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EXECUTIVE  SUMMARY 


This  simulation  tested  controllers'  ability  to  effectively 
resolve  conflicts  for  the  proposed  triple  simultaneous  instrument 
landing  system  (ILS)  approach  operation  at  the  new  Denver 
International  Airport  (DIA) .  A  model  of  DIA  with  triple 
simultaneous  parallel  approaches,  spaced  5280  and  7600  feet  (ft) 
apart,  and  a  field  elevation  of  5431  ft  at  an  ambient  temperature 
of  IS^C,  or  61**F,  was  incorporated  into  the  simulation. 
Controllers  used  Final  Monitor  Aid  (FMA)  displays  to  monitor 
final  approach  traffic.  The  radar  sensor  simulated  had  the 
performance  of  an  Airport  Surveillance  Radar  (ASR)-9  system 
enhanced  to  provide  improved  target  resolution  capabilities. 

Aircraft  blunders  were  used  to  test  the  controllers'  ability  to 
maintain  a  distance  of  500  ft  between  aircraft  during  critical 
situations.  A  blunder  occurred  when  one  aircraft  (i.e.,  a  Target 
Generation  Facility  (TGF)  aircraft) ,  established  on  an  ILS 
approach,  made  an  unexpected  turn  toward  an  aircraft  on  an 
adjacent  approach,  usually  a  flight  simulator.  Eighty  percent  of 
the  blunders  were  30-degree  turns  off  of  the  localizer,  17 
percent  were  20-degree  turns,  and  3  percent  were  10-degree  turns. 
Pilots  of  70  percent  of  the  blundering  aircraft  were  instructed 
to  disregard  controller  communications,  simulating  an  inability 
to  correct  the  blunder.  Statistical  analyses  evaluated  the  30 
degree,  "no-response"  blunders  that  were  initiated  into  flight 
simulator  targets.  A  test  criterion  violation  (TCV)  occurred 
when  the  separation  between  aircraft  was  less  than  500  ft. 
Blunders  that  would  have  resulted  in  an  aircraft  miss  distance  of 
less  than  500  ft  without  controller  intervention  were  classified 
as  "at  risk." 

Four  criteria  were  developed  by  the  Multiple  Parallel  Approach 
Program  (MPAP)  Technical  Work  Group  (TWG)  to  evaluate  the  study: 

1.  The  number  of  TCV's  relative  to  the  total  number  of  "at  risk" 
blunders ; 

2.  The  frequency  of  nuisance  breakouts  (NBO's)  and  No 
Transgression  Zone  (NTZ)  entries; 

3.  The  operational  assessments  from  participating  controllers, 
technical  observers,  and  MPAP  TWG  members;  and 

4.  A  risk  analysis  relative  to  one  fatal  accident  per  25,000,000 
approaches . 

During  the  simulation,  two  blunders  resulted  in  TCV's.  Analyses 
indicated  that  186  blunders  would  have  resulted  in  an  aircraft 
miss  distance  of  less  than  500  ft  without  controller 
intervention.  In  total,  controllers  were  able  to  successfully 
resolve  98.9  percent  of  all  "at  risk"  blunders. 
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An  evaluation  of  NBO's  and  NTZ  entries  was  conducted  to  assess 
total  navigation  system  error  (TNSE) .  NBO's  occurred  in  0.2 
percent  of  the  non-blunder  approaches.  In  addition,  the  DIA 
runway  spaclngs  were  sufficiently  large  that  aircraft  did  not 
enter  the  NTZ  as  a  consequence  of  TNSE. 

Participating  air  traffic  controllers  reported,  "In  this 
simulation  the  air  traffic  controller  team  believes  they  safely 
monitored  triple  simultaneous  ILS  approaches  at  the  simulated  new 
Denver  Airport  using  the  FMA.”  Technical  observers  unanimously 
agreed  that  controllers  "had  little  difficulty  detecting  and 
resolving  blunders."  It  was  also  noted  that,  "the  controllers 
seemed  so  comfortable  with  the  position  that  time  was  taken  to 
evaluate  options,  even  after  a  blunder  had  occurred." 

The  risk  assessment  indicated  the  operation  meets  the  target  risk 
of  1  fatal  accident  per  25,000,000  approaches. 

The  MPAP  TWG  evaluated  the  controllers  effectiveness  at  resolving 
conflicts,  the  frequency  of  NTZ  entries  and  NBO's,  and  the 
ability  of  the  system  to  maintain  the  predetermined  target  level 
of  risk  (1  fatal  accident  per  25,000,000  approaches).  Based  upon 
their  evaluations,  the  TWG  concluded  that  the  triple  simultaneous 
ILS  approach  operation  at  DIA  as  simulated  was  acceptable. 
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_ INTRQpyCTIPN. 


The  ability  of  the  National  Airspace  System  (NAS)  to  meet  future 
air  traffic  demands  has  been  a  serious  concern  at  the  national 
level.  With  the  growing  number  of  aircraft  in  the  NAS  and 
increasing  congestion  at  the  nation's  airports,  the  Federal 
Aviation  Administration  (FAA)  has  been  developing  programs  to 
improve  NAS  capacity  since  the  early  1980 's.  These  programs  have 
attempted  to  reduce  air  traffic  delays  by  meeting  the  increasing 
demands  on  the  NAS.  Programs  to  redesign  the  existing  airways 
structure  include  proposals  to  provide  a  more  modern  air  traffic 
flow  management  capability  and  major  programs  to  incorporate 
state-of-the-art  automation  technology  throughout  the  system. 

A  major  factor  influencing  system  capacity  has  been  the  number  of 
aircraft  that  can  land  at  an  airport  during  instrument 
meteorological  conditions  (IMC) .  Limitations  imposed  by  current 
airport  runway  configurations  and  associated  air  traffic 
separation  criteria  contribute  to  the  capacity  problem, 
particularly  as  they  relate  to  aircraft  executing  instrument 
landing  system  (ILS)  approaches  under  IMC.  Increases  in  the 
number  of  simultaneous  ILS  approaches  during  IMC  will  increase 
airport  capacity  and  potentially  improve  traffic  flow  throughout 
the  NAS. 

The  Denver-Stapleton  International  Airport  was  faced  with  the 
capacity  problems  mentioned  above,  as  well  as  noise  level  and 
environmental  concerns.  To  deal  with  these  issues  and  to  avoid 
potential  future  problems,  the  new  Denver  International  Airport 
(DIA)  is  being  constructed.  The  size,  location,  and  design  of 
the  new  airfield  will  help  overcome  capacity  issues,  such  as 
congestion  and  delays,  as  well  as  protect  surrounding 
environmental  interests.  DIA's  high  field  elevation,  however, 
will  affect  airport  and  aircraft  operations  in  many  ways, 
especially  with  respect  to  high  density  altitude.  High  density 
altitude  is  a  concern,  particularly  during  the  final  approach 
phase  of  flight,  as  it  affects  aircraft  performance. 

1.1  BACKGROUND. 

Density  altitude  has  been  defined  to  be  "a  measure  of  air  density 
used  to  determine  aircraft  performance”  (Jeppeson  Sanderson, 
1983).  Generally,  as  air  density  decreases  (i.e.,  as  a  function 
of  ambient  temperature,  altitude,  and  barometric  pressure) , 
density  altitude  increases.  Any  significant  increase  in  density 
altitude  can  result  in  drastic  reductions  in  engine  power  output, 
propeller  efficiency,  and  aerodynamic  lift  (Jeppeson  Sanderson, 
1990) .  This  degradation  of  performance  occurs  throughout  the 
airplane's  operational  envelope,  but  is  especially  noticeable  in 
takeoff  and  rate  of  climb  performance  (FAR/AIM,  1991) . 
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One  specific  effect  of  high  density  altitude  is  higher  true 
airspeed  (TAS) .  TAS  is  the  actual  speed  of  an  airplane  through 
the  air.  As  altitude  or  air  temperature  increases,  the  density 
of  the  air  decreases.  Indicated  airspeed  (IAS)  is  the  reading 
taken  directly  from  the  airspeed  indicator  on  an  airplane.  It 
does  not  reflect  variations  in  air  density  as  higher  altitudes 
are  reached.  For  a  given  IAS,  this  means  the  TAS  increases  with 
altitude  (Jeppeson  Sanderson,  1990) .  Aircraft  fly  the  same 
indicated  airspeed  at  both  high  density  and  sea  level  airports. 
Table  1  demonstrates  the  differences  between  indicated  airspeed 
and  true  airspeed  within  varying  high  density  altitudes. 


TABLE  1.  TRUE  AIRSPEED  VS.  INDICATE  AIRSPEED 


Txrte  Airspeed 

Indicated 

Airspeed 

Density  Altitude  (Feet) 

6000  9000  11,000 

130 

142 

149 

154 

140 

153 

160 

165 

150 

164 

172 

177 

160 

175 

183 

189 

170 

186 

195 

201 

180 

197 

206 

213 

190 

208 

218 

225 

200 

219 

229 

236 

210 

230 

240 

248 

220 

241 

252 

260 

If  an  airplane  on  approach  in  a  high  density  altitude  environment 
must  take  evasive  actions  to  avoid  a  blundering  aircraft,  the 
reduced  aircraft  performance  caused  by  higher  density  altitude 
could  become  a  critical  factor  in  its  ability  to  avoid  a 
collision.  High  density  altitude  affects  blunder  resolution  in 
two  ways.  First,  the  blundering  aircraft  has  a  higher  TAS, 
therefore,  a  higher  cross-track  velocity.  Second,  the  evading 
aircraft's  higher  TAS  would  result  in  a  larger  turn  radius, 
therefore,  a  decreased  ability  to  avoid  the  blundering  aircraft. 

If  controllers  could  immediately  detect  blunders,  there  would  be 
less  potential  for  a  conflict  to  occur.  The  high  resolution  and 
the  computer  generated  controller  alerts  of  the  Final  Monitor  Aid 
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(FMA)  display  were  designed  to  assist  controllers  in  the  early 
detection  and  resolution  of  blunders.  Studies  have  shown  that 
the  FMA  displays  provide  the  controllers  with  increased  time  to 
either  correct  the  course  of  a  blundering  aircraft  or  to  issue 
conflict  resolution  instructions  to  the  pilots  of  aircraft  on 
adjacent  parallel  approaches  (CTA,  1993  Report  in  progress) . 
Quicker  controller  response  times  available  with  the  FMA  will  be 
particularly  useful  in  counteracting  the  effects  of  high  density 
altitude. 

1.1.1  Multiple  Parallel  Approach  Program  fMPAPK 

The  FAA  established  the  MPAP  Technical  Work  Group  (TWG)  to 
evaluate  multiple  parallel  simultaneous  ILS  approaches.  Previous 
simulations  conducted  by  the  MPAP  have  evaluated  controller 
blunder  detection  and  conflict  resolution  performance  using  both 
the  Automated  Radar  Terminal  System  (ARTS)  and  the  Precision 
Runway  Monitor  (PRM)  System  (see  appendix  A) .  A  simulation 
completed  in  September  of  1990  (Phase  IV. b  of  the  MPAP)  examined 
triple  simultaneous  ILS  approaches  to  runways  spaced  5000  feet 
(ft)  apart,  near  sea  level  (i.e.,  field  elevation  of  600  ft), 
with  even  runway  thresholds.  Controllers  used  current  ARTS 
displays  and  a  simulated  Airport  Surveillance  Radar  (ASR)  system 
with  a  4.8  second  (s)  update  rate.  Findings  from  this  simulation 
indicated  these  operations  are  acceptable  near  sea  level.  The 
Phase  IV. b  study,  however,  did  not  address  the  effects  of  high 
density  altitudes  on  the  operation  of  multiple  parallel 
approaches . 

Using  the  above  simulation  as  a  baseline,  the  MITRE  Corporation 
conducted  analyses  that  evaluated  blunder  resolution  performance 
at  high  runway  elevations.  These  analyses  found  that  high 
airport/ runway  elevation  (5400  ft)  can  significantly  degrade 
blunder  resolution  performance.  An  empirical  examination  of  the 
effect  of  high  density  altitude  became  increasingly  important  as 
DIA,  with  a  field  elevation  of  5431  ft,  neared  completion. 

1.1. 1.1  Simulation  Using  the  ARTS  IIIA  Display. 

A  study  was  conducted  to  evaluate  controller  blunder  resolution 
performance  and  total  navigation  system  error  (TNSE)  (based  on 
the  frequency  of  nuisance  breakouts  (NBO's)  and  No  Transgression 
Zone  (NTZ)  entries) .  This  simulation  was  conducted  to  yield  data 
for  an  operational  assessment  and  a  risk  assessment  of  parallel 
operations  at  a  generic  high  density  altitude  airport.  This 
simulation  studied  simultaneous  ILS  approaches  to  both  triple  and 
quadruple  parallel  runways  at  a  field  elevation  (i.e.,  pressure 
altitude)  of  5431  ft.  Final  monitor  controllers  used  ARTS  IIIA 
displays  and  a  radar  with  a  4.8  second  update  rate. 

The  high  density  altitude  simulation  using  ARTS  IIIA  displays  was 
conducted  from  September  8  to  September  25,  1992,  at  the  FAA 
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Technical  Center.  Indicated  air  speeds  of  180  knots  (kn)  were 
assigned  for  turbojets,  150  kn  for  turboprops,  and  120  kn  for 
twin  engine  piston  aircraft.  Preliminary  data  indicated  that 
during  the  simulation,  a  total  of  746  blunders  were  initiated, 

376  for  the  triple  approach  configuration  and  370  for  the 
quadruple  approach  configuration.  Approximately  74  percent  of 
all  blundering  aircraft  did  not  respond  to  air  traffic  control 
( ATC )  commands . 

Preliminary  data  analyses  indicated  that  the  test  criterion 
violations  (TCV's)  (aircraft  separated  by  less  than  or  equal  to 
500  feet)  exceeded  the  test  criterion  rate  of  2  percent  set  to 
evaluate  the  operation.  However,  the  number  of  NBO's  and  NTZ 
entries  was  acceptable  based  on  the  criteria  set  by  the  TWG. 

Qualitative  data  from  the  controller  questionnaires  indicated 
that  when  the  ARTS  IIIA  display  was  used,  it  was  "very  difficult" 
to  perform  the  monitor  controller  task.  The  controllers  also 
disagreed  with  the  statement  that  "triple  independent  IFR 
approaches  to  runways  spaced  5280  and  7600  ft  apart  can  be  safely 
conducted  as  simulated." 

After  reviewing  the  preliminary  data,  controller  opinions,  and 
technical  observer  comments,  it  was  decided  that  an  additional 
high  density  altitude  simulation  should  be  conducted  using  the 
Full  Digital  ARTS  III  Display  System  (FDADS)  and  data  specific  to 
DIA.  To  make  this  simulation  site  specific  to  DIA,  additional 
information  about  air  traffic  and  airport  operations  was 
requested  from  representatives  from  the  Northwest  Mountain  Region 
and  the  Air  Traffic  Division  of  the  FAA. 

Itliltg _ Simulation  Using  the  FDADS. 

A  second  simulation  was  conducted  to  evaluate  the  performance  of 
controllers  monitoring  triple  simultaneous  approaches  to  parallel 
runways  using  the  FDADS.  In  this  simulation,  assigned  aircraft 
speeds  were  kept  at  170  kn  for  all  turbojet  and  turboprop 
aircraft.  Additionally,  twin-engine  piston  aircraft  were 
assigned  a  150  kn  IAS.  The  three-week  simulation  began  on 
November  16,  1992,  but  was  terminated  on  November  18,  1992, 
because  preliminary  data  indicated  that  the  operation  would  not 
meet  performance  goals  set  by  the  TWG. 

Preliminary  data  for  aircraft  separation,  NBO's,  and  NTZ  entries 
were  reviewed  daily.  At  the  end  of  the  second  day  of  simulation, 
1169  aircraft  were  handled,  87  blunders  were  initiated  into 
flight  simulators,  and  5  TCV's  were  reported.  Although  the  data 
for  NTZ  entries  and  NBO's  appeared  to  be  within  the  goals  set  by 
the  TWG,  this  sample  of  data  (5  TCV's  out  of  87  blunders  =  5.7 
percent  TCV  rate)  exceeded  the  blunder  resolution  performance 
goal  set  by  the  TWG.  Additionally,  TWG  members,  simulation 
participants,  and  observers  from  FAA  Headquarters  and  the 
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Northwest  Mountain  Region  also  agreed  that  the  operation,  as 
simulated  (i.e.,  using  the  FDADS) ,  would  not  meet  the  evaluation 
criteria.  Documents  supporting  this  decision  can  be  found  in 
appendixes  B,  C,  and  D. 

Beginning  November  18,  1992,  the  remainder  of  the  simulation  was 
conducted  using  the  FMA  with  a  radar  update  rate  of  4.8  seconds. 
The  FMA  was  used  because  the  high  resolution  color  display  and 
the  automated  visual/aural  alarms  were  shown  to  improve  blunder 
resolution  performance  (Fabrizi,  M. ,  Massimini,  S.,  and  Toma,  N. , 
1993)  . 

2.  SIMULATION  DESIGN. 

This  study  was  a  real-time  ATC  simulation  to  evaluate  final 
approach  operations  at  DIA.  Controllers  used  FMA  displays  to 
monitor  triple  simultaneous  parallel  ILS  approaches  under  IMC. 

The  following  sections  will  discuss  the  parameters,  equipment, 
personnel,  and  procedures  used  for  the  simulation. 

2 . 1  SIMULATION  PARAMETERS . 

In  order  to  assess  the  viability  of  the  triple  parallel  runway 
configuration,  certain  operational  parameters  were  defined  and 
included  in  the  DIA  simulation.  Parameters,  including  the 
pressure  altitude,  temperature,  aiirport  configuration,  and 
aircraft  mix,  were  derived  from  information  provided  by  the 
Northwest  Mountain  Region. 

_ Density  Altitude. 

As  explained  in  earlier  sections,  DIA  will  be  located  in  a  region 
where  high  density  altitude  can  be  a  problem  for  aircraft  on 
approach.  To  account  for  the  potentially  dramatic  effect  of 
high  density  altitudes  on  aircraft  performance,  all  aircraft  in 
this  simulation  were  programmed  to  perfoxm  at  a  pressure  altitude 
of  5431  ft  on  a  16®C  day,  which  is  equivalent  to  a  6500  ft  mean 
sea  level  (M.S.L.)  altitude  at  standard  temperature  and 
pressure.  The  temperature  was  based  on  a  review  of  the  90th 
percentile  surface  temperature  during  instrument  operations  at 
Denver-Stapleton  Airport. 

2.1.2  Airport  Conf icruration  and  Flight  Patterns. 

DIA  was  modeled  with  three  12,000-ft  parallel  runways  with 
staggered  thresholds  (figure  1).  Runways  17L  and  17R  were  spaced 
5280  ft  apart,  while  the  17R  and  16  runways  were  spaced  7600  ft 
apart  (centerline  to  centerline) .  Outer  markers  (OM) ,  turn-on 
altitudes,  and  glide  slope  intercepts  (GSI)  for  each  runway  are 


5 


i 

m 

} 
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GSI  -  glide  slope  intercept 
Field  Elevation  5431  feet 


Note:  Drawing  not  to  scale 


FIGURE  1.  AIRPORT  CONFIGURATION 
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presented  in  table  2.  Aircraft  executed  30-degree  localizer 
intercepts  approximately  3  nautical  miles  (nml)  from  their 
respective  GSI  point. 


TABLE  2.  DIA  ILS  RUNWAY  TURN-ON  ALTITUDES 


1  Runway 

Outer  Marker 
(nmi) 

Turn-On 
Altitude  (ft) 

Glide  Slope 
Intercept  (nmi) 

16 

4.5 

10,000 

14.36 

17R 

4.9 

11,000 

17.5 

17L 

4.8 

9,000 

11.21 

Although  minor  differences  between  touchdown  zone  elevations  are 
planned  for  the  runways  at  DIA,  the  simulation  used  a  common 
touchdown  zone  altitude  for  all  three  runways  because  of  cui.r»  it 
limitations  in  the  simulation  capabilities  of  the  Target 
Generation  Facility  (TGF) .  GSI  points  also  varied  from  the 
actual  DIA  operation  due  to  the  common  touchdown  zone  elevation. 
The  MPAP  TWG  and  representatives  from  the  Northwest  Mountain 
Region  and  DIA  concurred  that  the  differences  were  minimal  and 
would  not  affect  controller  performance. 

Approach  plates  were  produced  based  upon  the  simulated  airport 
configuration  (see  appendix  E) .  The  approach  plates  included 
runway  layouts,  spacings,  and  arrival  frequencies.  All  flight 
simulator  approaches  in  the  simulation  were  made  using  these 
approach  plates. 

2.1.3  Blunders . 

Aircraft  blunders  were  used  to  test  the  controllers'  ability  to 
maintain  a  500  ft  distance  between  aircraft  during  critical 
situations.  During  each  run  of  the  simulation,  blunders 
transpired  without  warning  to  the  controllers.  A  blunder 
occurred  when  one  aircraft,  established  on  the  ILS  localizer 
approach,  made  an  unexpected  turn  towards  a  second  aircraft  on  an 
adjacent  approach  course. 

In  normal  operations,  the  controller  detects  the  deviation  of  the 
blundering  aircraft,  issues  instructions  to  resolve  the 
situation,  and  the  pilot  verbally  responds  and  complies  as 
instructed.  However,  in  some  situations,  the  blundering  aircraft 
may  be  unable  to  correct  its  deviation  and  may  pass  through  the 
NTZ,  and  cross  the  adjacent  approach  course.  This  inability  to 
correct  the  blunder  can  arise  from  several  causes  (e.g., 
communications,  hardware  failure,  human  error,  etc.).  To 
simulate  this  situation,  pilots  of  blundering  aircraft  were 
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sometimes  instructed  to  disregard  controller  communications, 
thereby  not  correcting  the  blunder.  A  blunder  with  this 
condition  will  be  referred  to  as  a  "no-response"  blunder  in  this 
report . 


2. 1.3.1  _ Blunder  Scripts. 

Blunder  scripts  were  developed  from  the  traffic  samples  to  assist 
the  test  director  with  creating  potential  TCV's.  All  blunders 
were  initiated  by  TGF  aircraft  and  occurred  after  vertical 
separation  had  been  lost  with  aircraft  on  an  adjacent  approach. 
Sixty  percent  of  the  blunders  were  scripted  to  occur  between  the 
17L  and  17R  runways,  with  the  remaining  40  percent  occurring 
between  runways  17R  and  16. 

Eighty  percent  of  the  blunders  were  scripted  to  be  30-degree 
turns  off  of  the  localizer,  17  percent  were  scripted  to  be  20- 
degree  turns,  and  3  percent  of  the  blunders  were  scripted  to  be 
10-degree  turns.  In  order  to  simulate  worst  case  scenarios,  70 
percent  of  the  blunders  were  scripted  as  no-response  blunders. 

Only  30  degree,  no-response  blunders  initiated  into  flight 
simulator  targets  were  assessed  in  the  statistical  evaluation  of 
the  data.  In  previous  simulations,  controllers  have  been  able  to 
resolve  10  and  20 -degree  turns,  and  these  blunders  only 
contributed  to  about  1  percent  of  the  total  risk. 

2. 1.3. 2  Closest  Point  of  Approach  fCPAl  Prediction  Tool. 

The  CPA  Prediction  Tool  is  a  software  tool  used  by  the  test 
director  in  creating  potential  TCV's.  The  software  presented  the 
call  signs  of  the  blundering  and  the  evading  aircraft  in  a  window 
on  the  test  director's  display.  For  each  aircraft  pair,  the  CPA 
Prediction  Tool  used  aircraft  velocities,  headings,  and  blunder 
degree  in  the  real-time  calculation  of  a  predicted  CPA.  The  time 
until  the  CPA  would  be  reached,  given  an  inonediate  execution  of 
the  blunder,  was  also  calculated.  This  information  was  updated 
with  each  radar  update,  every  4.8  seconds,  and  was  presented  with 
the  aircraft  call  signs. 

The  window  had  the  capacity  to  accommodate  four  aircraft  pairs  at 
one  time.  The  aircraft  pairs  which  appeared  in  the  window  were 
determined  by  the  scripted  blunder  scenarios;  however,  the  test 
director  had  the  capability  to  create  blunders  that  were  not 
designated  on  the  blunder  scripts.  The  test  director  also  had 
the  capability  to  delete  aircraft  pairs  from  the  window. 

_ Traffic  Samples. 

The  traffic  samples  were  based  on  actual  arrival  traffic  into 
Denver-Stapleton  Airport.  Each  of  the  samples  was  composed  of  a 
representative  population  of  propeller-driven,  turboprop,  and 
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turbojet  aircraft.  Sixty-four  percent  of  each  traffic  sample 
were  air  carriers,  30  percent  were  commuters,  and  approximately  6 
percent  of  each  sample  were  general  aviation  aircraft. 

All  turbojet  and  turboprop  aircraft  were  assigned  an  initial 
approach  speed  of  170  kn  IAS,  plus  or  minus  10  kn.  Twin-engine 
piston  aircraft  were  assigned  a  150  kn  IAS,  plus  or  minus  10  kn. 
Additionally,  each  traffic  sample  included  two  to  three  speed 
overtakes  during  each  run  to  provide  additional  realism  to  the 
controllers. 

2.1.5  TNSE  Model. 

Aircraft  position,  with  respect  to  the  extended  runway 
centerline,  the  NTZ,  and  to  other  aircraft,  must  be  realistically 
presented  on  the  radar  display  to  accurately  assess  the 
controllers'  ability  to  detect  a  blunder.  In  developing  the 
navigational  error  model  for  TGF  aircraft,  two  criteria  were 
used.  First,  aggregate  errors  must  accurately  reflect  the  TNSE 
distribution  of  aircraft  as  they  fly  ILS  approaches  in  the 
operational  environment.  Second,  displayed  flight  paths  of 
aircraft  must  look  reasonable  to  the  controllers;  i.e., 
deviations  from  the  localizer  centerline  should  appear  to  be 
typical  of  aircraft  flying  an  ILS  approach  during  IMC.  The 
navigational  error  model  used  for  this  simulation  was  based  upon 
data  collected  at  Chicago  O'Hare  International  Airport  (ORD) 
(Timoteo,  D.  and  Thomas,  J.,  1989),  Memphis  International  Ai]:i>ort 
(HEM)  (PRM  Program  Office,  1991),  Los  Angeles  International 
Airport  (LAX)  (DiMeo,  K.  et  al,  1993,  Report  in  progress),  and 
data  collected  at  the  FAA  Technical  Center  investigating  TNSE  at 
high  density  altitudes  using  flight  simulators. 

Review  of  the  flight  tracks  collected  at  ORD  and  MEM  indicated 
that  the  TNSE  generally  consisted  of  a  combination  of  two 
elements.  First,  the  aircraft  often  flew  a  course  which  was 
asymptotic  with  the  actual  runway  centerline  extended.  Second, 
about  this  course  there  were  often  cyclic  and  periodic 
deviations.  To  simulate  this  pattern  of  flight,  a  system  using 
"pseudoroutes"  and  "fans"  about  the  pseudoroutes  was  developed. 
Further,  preliminary  LAX  data  supports  the  assumption  of 
linearity  of  the  TNSE  model  out  to  20  nmi. 

As  shown  in  figure  2a,  a  pseudoroute  was  defined  to  be  a  straight 
line  which  simulated  the  asymptotic  element  of  navigational 
error.  It  began  at  the  center  of  the  runway  threshold  and 
extended  outward  beyond  20  nmi.  Starting  at  the  runway 
threshold,  pseudoroutes  were  offset  from  the  ILS  localizer 
centerline  based  upon  a  normal  distribution  with  a  mean  of  0 
degrees  and  a  standard  deviation  of  approximately  0.29  degrees. 

A  fan-shaped  envelope  was  added  on  either  side  of  the  pseudoroute 
to  accurately  represent  the  deviations  around  the  pseudoroute. 
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As  seen  in  figure  2b,  the  fan  began  at  the  runway  threshold  and 
was  bisected  by  the  pseudo-route.  As  aircraft  traveled  along  the 
pseudoroute,  they  flew  between  the  fan  boundaries.  Individual 
aircraft  began  a  half-standard  rate  turn  (1.5  degrees  per  second) 
towards  the  opposite  fan  boundary  after  coming  within  plus  or 
minus  2  seconds  time  of  the  fan  boundary.  As  the  aircraft 
approached  the  opposite  fan  boundary,  it  once  again  made  a  slow 
turn  back  towards  the  first  boundary.  This  process  was  repeated 
throughout  the  approach  until  the  aircraft  landed  or  was  given  a 
heading  change  by  the  controller. 

The  resultant  flight  paths  satisfied  the  first  criteria  by 
producing  navigational  error  distributions  that  corresponded 
closely  with  those  found  in  previous  data.  In  addition,  the  new 
flight  paths  also  satisfied  the  second  requirement  since  they 
provided  realistic  visual  targets  for  the  controllers. 

2t.lt  6 _ Radar  Error. 

The  representation  of  aircraft  positions  on  controllers'  displays 
may  differ  from  the  actual  aircraft  position  due  to  inaccuracies 
in  the  radar  system.  Therefore,  to  accurately  represent  the 
operational  environment,  radar  error  was  included  as  a  parzuneter 
in  the  DIA  simulation.  The  radar  error  model  for  this  simulation 
as  based  upon  the  performance  characteristics  of  an  ASR-9  system 
enhanced  to  provide  improved  target  resolution  capabilities.  The 
range  error  had  a  mean  egual  to  zero  and  a  standard  deviation  of 
185  ft. 

The  DIA  simulation  assumed  a  radar  system  with  2.7  milliradians 
azimuth  accuracy  or  better,  and  the  capability  to  resolve  two 
aircraft  at  20  nmi  separated  by  0.9  degrees  or  more.  Thus,  the 
radar  requirements  were:  1)  normal  azimuth  accuracy  of  2.7 
milliradians,  and  2)  resolution  (resolving  power)  of  2  aircraft 
targets  at  the  same  range  (20  nmi)  separated  by  0.9  degrees  or 
more.  The  second  requirement  was  equivalent  to  resolution  of  2 
targets  slightly  more  than  2000  feet  apart  at  20  nmi. 

2.2  EQUIPMENT. 

This  section  describes  the  equipment  that  was  used  during  the 
simulation. 

2.2.1  TGF. 

The  TGF  is  an  advanced  simulation  system  designed  to  support 
testing  of  current  and  future  ATC  systems  at  the  FAA  Technical 
Center.  The  functionality  of  the  TGF  system  is  partitioned  in 
three  siibsystems:  Simulation  Pilot,  Target  Generation,  and 
Development  and  Support. 
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The  Slnulatlon  Pilot  Subsystem  consists  of  both  the  Simulation 
Pilot  Workstations  (SPW's)  and  the  Exercise  Control  Workstations 
(ECW's) .  Each  workstation  consists  of  a  386-based  personal 
computer,  running  under  DOS.  The  SPW's  are  mounted  in  pairs  to  a 
customized  table,  which  contains  a  communication  system  that 
provides  an  audio  interface  with  air  traffic  controllers.  The 
Simulation  Pilot  Operators  (SPO's)  use  the  SPW's  to  "fly"  the 
simulated  aircraft  and  command  them  according  to  ATC 
instructions . 

The  Target  Generation  Siibsystem  consists  of  both  a  Target 
Generation  (TG)  chassis  and  External  Interface  (El)  chassis. 

Each  chassis  is  based  upon  a  VME  architecture  employing  68030 
processor  boards.  The  TG  performs  all  modeling  within  the  TGF 
and  correlates  dynamic  data  such  as:  aircraft  state  vectors, 
radar  performance,  weather  vectors  and  states,  with  known  flight 
plan  and  adaptable  data.  The  El  is  responsible  for  creating  the 
exact  form  and  content  of  the  digitized  radar  messages  sent  to 
the  ATC  system  under  test.  Controller-pilot  voice  communications 
are  processed  through  an  AMECOM  voice  communications  system. 

The  Development  and  Support  Subsystem  is  based  on  a  SUN 
architecture  that  employs  a  SUN  3/470  as  a  file  server,  a  set  of 
peripherals,  and  the  Sun  3/80  diskless  computer  as  the 
Development  and  Support  Workstation  (DSW's) .  The  Development  and 
Support  Subsystem  provides  the  basic  post-exercise  data  reduction 
and  analysis  capabilities.  In  addition,  this  subsystem  provides 
the  capabilities  necessary  to  maintain  and/or  enhance  the  TGF 
software . 

In  total,  the  TGF  models  a  logical  view  of  the  ATC  environment, 
including  long  and  short  range  radar  sensors,  controlled 
airspace,  weather  conditions,  air  traffic,  and  aircraft 
performance.  The  TGF  configuration  in  respect  to  this  simulation 
is  shown  in  figure  3. 

2.2.2  FMA  Displays . 

A  digital  video  map  of  DIA  was  presented  to  controllers  on  three 
FMA  displays  located  in  the  Systems  Display  Laboratory  at  the  FAA 
Technical  Center.  The  FMA  is  a  high  resolution  color  display 
that  is  equipped  with  the  controller  alert  system  hardware  and 
software  which  is  used  in  the  PRM  system.  The  display  includes 
alert  algorithms  providing  the  target  predictors,  a  color  change 
alert  when  a  target  penetrates  or  is  predicted  to  penetrate  the 
NTZ,  a  color  change  alert  if  the  aircraft  transponder  becomes 
inoperative,  synthesized  voice  alerts,  digital  mapping,  and  like 
features  contained  in  the  PRM  system. 

The  graphics  for  the  monitors  were  generated  by  a  Metheus 
graphics  driver,  and  the  display  system  was  driven  by  a  micro-VAX 
computer.  In  addition  to  the  mapping  information  currently 
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provided  by  ARTS  displays,  FMA's  provide  controllers  with 
features  to  aid  them  in  the  early  detection  of  blunders  and  the 
control  of  airspace.  These  include  independent  axes  expansion 
capabilities,  color  coding,  aircraft  predictor  lines,  and  audio 
and  visual  warnings. 

With  FMA's,  vertical  and  horizontal  (Y-X)  axes  can  be  expanded 
Independently,  in  accordance  with  site  variable  requirements,  to 
improve  the  controller's  ability  to  detect  aircraft  movement  away 
from  the  extended  runway  centerline.  For  this  simulation,  the 
ratio  for  the  horizontal  axis  was  6  times,  while  the  vertical 
axis  was  1.5  times  on  the  controllers'  displays. 

For  each  of  the  three  runways,  ILS  approach  centerlines  were 
displayed  as  dashed  white  lines,  where  each  dash  and  each  space 
between  dashes  were  scaled  to  represent  1  nmi.  Additional  solid 
light  blue  lines  were  on  each  side  of  the  ILS  centerline  to 
delineate  200  foot-deviations  from  the  localizer.  The  2000-ft 
wide  NTZ,  between  extended  runway  centerlines,  was  outlined  in 
red. 

A  predictor  line  was  used  in  the  generation  of  the  audio  and 
visual  alerts.  The  predictor  line,  which  was  affixed  to  each 
aircraft  target,  indicated  where  the  aircraft  would  be  in  10  s  if 
it  continued  on  the  same  path.  The  predictor  line  provided  the 
controller  with  advance  notice  of  the  path  of  the  aircraft.  The 
predictor  line  can  be  varied,  but  for  this  simulation  it  was  set 
to  10  s. 

Aircraft  targets  and  alphanumeric  data  blocks  were  presented  in 
green,  as  long  as  they  maintained  an  approach  within  the  normal 
operating  zone  (NOZ) .  When  the  predictor  line  indicated  that  an 
aircraft  was  within  10  s  of  entering  the  NTZ,  the  green  aircraft 
target  and  data  block  changed  to  yellow.  An  auditory  warning 
also  sounded  (e.g.,  "American  211")  to  notify  the  controller  of 
the  impending  NTZ  entry.  If  the  aircraft  entered  the  NTZ,  the 
yellow  aircraft  target  and  data  block  immediately  changed  to  red. 

2.2.3  Flight  Simulators  and  Simulator  Parameters. 

Six  Part  121  aircraft  simulators  and  one  general  aviation  trainer 
(GAT)  were  integrated  into  the  simulation.  This  group  included 
simulators  from:  NASA-Ames,  Moffett  Field,  CA;  AVIA  Inc.,  Costa 
Mesa,  CA;  Boeing  IiiC. ,  Seattle,  WA;  Trans  World  Airlines,  St. 
Louis,  MO;  Delta  Airlines  Inc.,  Atlanta,  GA,  and  the  FAA 
Technical  Center,  Atlantic  City  International  Airport,  NJ. 

Having  the  flight  simulators  as  an  integral  part  of  the 
simulation  increased  the  validity  of  the  findings  by  providing  a 
representative  sample  of  NAS  users  (i.e.,  currently  licensed 
airline  pilots  who  staffed  the  simulators) .  It  also  generated 
more  accurate  data  with  respect  to  aircraft  and  pilot 
performance . 
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Flight  simulators  assumed  the  configuration  of  aircraft  flying 
the  localizer  course  and  replaced  TGF  aircraft  that  were 
scheduled  to  enter  the  traffic.  Crosswinds  were  Introduced  to 
flight  simulator  approaches  to  provide  pilots  with  a  realistic 
flight  environment.  Three  wind  conditions  were  assigned:  no 
wind,  wind  from  the  east,  or  wind  from  the  west.  Winds  were 
started  at  25  kn  and  decreased  to  10  kn  at  the  outer  marker. 
Flight  simulator  approaches  were  flown  using  autopilot,  flight 
director,  or  raw  data,  in  accordance  with  an  individual  captain's 
discretion  and  his  respective  company  policy. 

2^ _ SIMULATION  PERSONNEL. 

The  following  section  describes  the  various  personnel  Involved  in 
this  simulation  and  their  responsibilities. 

2.3.1 _ Controllers. 

Ten  air  traffic  controllers  with  experience  in  multiple  parallel 
approaches  participated  as  test  subjects  in  the  DIA  simulation. 
Six  of  the  controllers  were  selected  from  various  Terminal  Radar 
Approach  Control  (TRACON)  facilities  across  the  country,  and  four 
positions  were  filled  by  controllers  from  the  existing  Denver 
TRACON  facility.  Two  of  the  Denver  controllers  participated  the 
first  week,  and  the  other  two  participated  the  remainder  of  the 
simulation. 

Controllers  staffed  the  three  final  approach  monitor  positions. 
They  monitored  the  flight  path  of  the  aircraft  on  their  assigned 
runway  and  ensured  aircraft  maintained  the  required  separation. 

In  the  event  of  a  blunder  or  an  NTZ  penetration,  controllers 
issued  the  appropriate  control  instructions  to  resolve  the 
conflict  situation.  When  controllers  were  not  working  the 
monitor  position,  they  were  often  reassigned  to  perform  the  local 
tower  control  function  in  an  attempt  to  generate  realistic 
communications  on  the  approach  frequencies. 

Individual  controllers  were  scheduled  to  work  as  a  monitor 
controller  for  half  of  each  2-hour  run.  A  controller  rotation 
period  was  scheduled  at  the  midpoint  of  each  two-hour  run  to 
simulate  actual  work  rotations  and  to  give  monitor  controllers  a 
rest.  Blunders  were  scripted  to  occur  at  any  time  during  a  run, 
including  during  the  controller  rotation  periods.  Controllers 
were  not  scheduled  to  work  the  monitor  position  for  more  than  2 
consecutive  hours  at  a  time. 

Monitor  controllers  completed  a  questionnaire  after  each  run  and 
at  the  conclusion  of  the  simulation.  After  each  run  in  which  a 
potential  TCV  occurred,  the  controllers  who  monitored  the 
blundering  and  evading  aircraft  completed  a  TCV  Statement.  These 
questionnaires  can  be  found  in  appendix  F. 
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Controllers  also  svibinitted  a  report  documenting  their  experiences 
during  the  simulation.  The  controller  report  is  included  as 
appendix  H  of  this  report. 

2^3 12 _ Simulation  Pilot  Operators. 

SPO's  operated  the  TGF  aircraft  within  the  various  traffic 
scenarios.  Throughout  the  simulation,  SPO's  responded  to 
controller  instructions  by  entering  aircraft  heading  and  altitude 
changes  using  their  specialized  computer  keyboard  and  display. 
SPO's  performed  their  task  from  the  Simulation  Pilot  Complex  in 
the  TGF  at  the  FAA  Technical  Center. 

2.3.3  _ Test  Director. 

Simulation  runs  and  aircraft  blunders  were  under  the  direction  of 
the  simulation  test  director.  Three  individuals  assumed  the  role 
of  test  director  throughout  the  simulation.  These  individuals 
have  extensive  ATC  experience  and  were  trained  to  work  with  the 
CPA  Prediction  Tool.  The  test  director  was  responsible  for 
initiating  blunders  based  upon  the  information  provided  by  the 
blunder  scripts,  the  CPA  Prediction  Tool,  and  his  expert 
judgment. 

2.3.4  _ Technical  Observers. 

Four  technical  observers  participated  in  the  DIA  simulation.  All 
technical  observers  had  past  ATC  experience,  and  some  had  FAA 
supervisory  experience.  All  technical  observers  were  familiar 
with  the  current  MPAP  project.  Technical  observers  monitored 
controller  actions  during  each  simulation  run.  Their  duties 
included  documenting  discrepancies  between  issued  control 
instructions  and  actual  aircraft  responses;  assisting  with 
alerting  responsible  parties  to  correct  any  problems  that  may 
have  occurred  during  the  test  (e.g.,  computer  failure,  stuck 
microphone) ;  assisting  controllers  with  the  preparation  of 
incident  reports;  and  preparing  a  Technical  Observer  Report  at 
the  end  of  the  simulation.  This  report  included  their  opinions 
and  conclusions  concerning  the  conduct  of  the  simulation  as  well 
as  any  recommendations  to  the  MPAP  TWG.  The  report  is  included 
as  appendix  I  to  this  report. 

2.3.5  Simulation  Observers . 

Simulation  observers  recorded  information  from  the  simulation, 
such  as  occurrences  of  blunders,  NBO's,  lost  beacon  signals 
(i.e.,  aircraft  that  went  into  coast),  and  system  problems  (e.g., 
hardware/software  failure,  communications,  etc.).  One  simulation 
observer  was  stationed  in  the  Systems  Display  Laboratory,  and 
another  was  located  in  the  area  of  the  test  director. 
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2.3.6  Site  Coordinators. 


A  site  coordinator  was  assigned  to  each  flight  simulator  location 
to  coordinate  efforts  with  "^.he  test  director  at  the  FAA  Technical 
Center  and  to  support  pilots  during  their  participation  in  the 
simulation.  Site  coordinators  acted  as  observers  and  did  not  aid 
the  aircrews  during  their  approaches.  Their  responsibilities 
included  briefing  aircrews,  providing  pilots  with  flight 
information  prior  to  each  approach,  documenting  approach 
information,  and  administering  questionnaires  to  the  pilots. 

2.3.7  Flight  Simulator  Pilots. 

Sixty  current  air  carrier  and  air  taxi  pilots  were  assigned  to 
fly  the  flight  simulators.  Two  pilots  were  assigned  to  each 
flight  simulator  during  each  run  of  the  simulation.  Pilots 
rotated  between  roles  (i.e.,  captain,  first  officer,  observer) 
throughout  the  day. 

2.4  SIMULATION  PROCEDURES. 

The  simulation  was  conducted  November  18  through  November  20  and 
November  30  through  December  17,  1992.  Three,  2-hour  runs  were 
scheduled  for  each  day.  During  the  final  week  of  the  simulation, 
an  additional  1-hour  run  was  conducted  daily.  Practice  trials 
were  scheduled  for  the  entire  first  run  on  November  18  and  the 
first  hour  of  the  first  run  on  November  30  to  acquaint 
controllers  with  the  displayed  triple  approach  operations  and  the 
radar/FMA  configuration.  Runs  were  not  scheduled  for  Saturdays 
and  Sundays. 

Controllers  staffed  their  positions  and  issued  appropriate 
control  instructions  to  maintain  separation  between  blundering 
and  evading  aircraft.  With  the  exception  of  the  pilots  of 
blundering  aircraft  during  no-response  blunders,  pilots  would 
verbally  respond  and  comply  as  instructed.  Aircraft  that 
blundered,  or  were  vectored  off  their  ILS  as  a  result  of  a 
blunder,  were  removed  from  the  traffic. 

3.  DATA  ANALYSIS. 

This  section  discusses  the  various  qualitative  and  descriptive 
statistical  approaches  that  were  used  to  analyze  the  DIA 
simulation  data. 

3.1  DATA  COLLECTION. 

The  controllers'  ability  to  resolve  blunders,  including  factors 
that  potentially  affected  their  performance,  was  examined  using 
descriptive  statistical  and  qualitative  analyses.  Data  files 
included  the  following; 
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a.  NTZ  transgression  frequency; 

b.  NBO  frequency; 

c.  Parallel  conflict  frequency  (i.e.,  conflicts  between 
aircraft  on  different  approach  courses) ;  and 

d.  Parallel  conflict  slant  range  miss  distances. 

Any  conflict  that  resulted  in  a  CPA  less  than  500  ft  was 
considered  a  TCV  and  was  investigated  to  determine  its 
operational  Impact.  A  comprehensive  review  of  TCV's,  including 
audio/visual  information,  controller-pilot  communications,  and 
computer  data,  was  conducted  to  ascertain  whether  any  single 
factor  contributed  to  the  severity  of  the  conflict. 

The  communication  frequencies  of  the  blundering  and  evading 
aircraft  and  visual  components  of  each  run  were  recorded  on  a 
Super-VHS  video  cassette  recorder.  Complete  audio  recordings 
were  made  using  a  20-channel  DICTAPHONE  audio  recorder  and  a  9- 
channel  lONICA  audio  recorder.  Both  the  DICTAPHONE  and 
the  lONICA  systems  were  recorded  from  the  AMECOM  system  and 
operated  independent  of  one  another  and  the  TGF  operating  system. 

3.1.1 _ Questionnaires  and  Observer  Loos. 

After  controllers  participav ed  as  a  monitor  controller,  they 
completed  a  Post-Run  Controller  Questionnaire.  This 
questionnaire  addressed  the  level  of  activity,  stress,  and  mental 
effort  they  experienced  during  the  run.  When  a  TCV  occurred, 
controllers  described  the  incident  on  a  TCV  statement. 

Controllers  also  completed  a  Post-Simulation  Questionnaire  at  the 
conclusion  of  their  participation  in  the  simulation.  The  Post- 
Simulation  Questionnaire  addressed  the  operational  viability  of 
the  display,  the  equipment,  and  the  runway  configuration.  A  copy 
of  these  questionnaires  can  be  found  in  appendix  F. 

Technical  observers,  simulation  observers,  and  site  coordinators 
each  recorded  information  using  logs  designed  specifically  for 
their  task.  In  general,  the  logs  were  devised  to  permit 
observers  to  record  information  pertaining  to  blunders,  TCV's, 
NBO's,  simulation  problems,  and  the  like,  without  being 
distracted  from  their  task.  Site  coordinator  logs  were  developed 
more  specifically  to  record  approach  information.  A  copy  of 
these  logs  can  be  found  in  appendix  J.  Information  provided  by 
these  logs  was  used,  along  with  the  controller  questionnaires,  to 
support  computer  data  files. 

3.2  ACCEPTANCE  CRITERIA. 

This  simulation  evaluated  the  ability  of  controllers  using  FMA 
displays  to  monitor  final  approach  traffic  at  DIA.  As  stated  in 
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section  2,  four  criteria  were  selected  by  the  MPAP  TWG  to 
evaluate  the  operations  that  were  simulated: 

a.  The  number  of  TCV's  relative  to  the  total  number  of  "at 
risk"  blunders,  with  a  goal  of  2  percent; 

b.  The  frequency  of  NBO's  and  NTZ  entries; 

c.  The  operational  assessments  from  participating 
controllers,  technical  observers,  and  MPAP  TWG  members;  and 

d.  A  risk  analysis  relative  to  one  fatal  accident  per 
25,000,000  approaches. 

3.3  SAMPLE  SIZE  AND  STATISTICAL  TECHNIQUES. 

3.3.1  "At  Risk"  Blunders. 

Controller  performance  was  measured  by  determining  the  proportion 
of  successfully  resolved  conflicts  relative  to  the  blunders  that 
would  have  resulted  in  a  TCV.  Specifically,  the  TWG  was 
concerned  with  non-responding  blunders  toward  flight  simulator 
targets  that  would  have  resulted  in  a  TCV  if  the  controllers  did 
not  intervene.  These  blunders  were  identified  and  classified  as 
being  "at  risk." 

Two  methods  were  used  to  determine  whether  a  blunder  was  "at 
risk"  for  the  simulation  runs.  The  first  method,  no-controller 
intervention  runs,  were  conducted  prior  to  the  simulation.  In 
these  runs,  controllers  did  not  participate,  and  if  a  blunder 
resulted  in  a  TCV,  it  was  classified  as  "at  risk."  No-controller 
intervention  runs  were  conducted  to  determine  the  probability  of 
"at-risk"  blunders  during  the  simulation.  These  data  were  used 
to  estimate  the  total  number  of  runs  which  needed  to  be  executed 
to  produce  a  sufficient  sample  size. 

Due  to  variations  in  aircraft  speeds  and  blunder  timing,  an  "at 
risk"  categorization  for  a  specific  blunder  may  vary  between 
simulation  runs.  Therefore,  a  second  method  was  used  in  the 
analysis  of  the  simulation  data. 

The  aircraft  position  data  were  analyzed  post-hoc  to  determine 
whether  the  blunder  was  indeed  "at  risk."  Aircraft  position  data 
and  speed  data  at  the  beginning  of  the  blunder  were  used  to 
generate  aircraft  tracks  as  if  the  controller  did  not  intervene. 
This  provided  an  accurate  method  for  determining  "at  risk" 
because  actual  position  and  speed  data  of  the  blundering  and 
evader  aircraft  were  used  in  the  determination. 
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3. 3. 1.1  No-Controller  Intervention  Runs. 


To  determine  the  number  of  “at  risk"  blunders  in  the  simulation, 
five  no-controller  intervention  runs  were  conducted  November  23 
through  November  25,  1992.  Traffic  samples  from  the  simulation 
were  run,  and  blunders  were  Initiated  without  controllers  present 
to  maintain  separation.  The  30-degree  blunders  that  resulted  in 
a  TCV  were  used  to  estimate  the  number  of  “at  risk"  blunders  in 
the  simulation. 

Descriptive  analyses  were  conducted  for  the  five  no-controller 
intervention  runs.  Overall  there  were  1225  approaches.  There 
were  approximately  38  approaches  per  runway  per  hour.  Of  the  167 
blunders  Initiated,  101  were  30-degree  blunders.  The  30-degree 
blunders  resulted  in  23  TCV's.  Thus,  the  test  director  created 
conflicts  with  CPA's  less  than  500  ft  in  an  average  of  22.8 
percent  of  all  blunders.  This  “at  risk"  percentage  was 
considered  a  gross,  conservative  estimate  used  to  calculate  the 
number  of  blunders  to  initiate  during  the  simulation. 

3. 3. 1.2  Post-Hoc  "At  Risk"  Categorization. 

For  every  aircraft  in  the  simulation,  data  were  collected  for  its 
X,  Y,  and  Z  position  on  a  second-by-second  basis.  Post-hoc 
analysis  of  this  data  provided  an  accurate  measure  of  whether  the 
aircraft  pair  would  have  resulted  in  a  TCV  had  the  controllers 
not  intervened. 

Analysis  of  the  X,  Y,  and  Z  aircraft  position  data  for  the 
simulation  runs  indicated  that  27  percent  of  all  30-degree,  no¬ 
response  blunders  into  flight  simulators  would  have  resulted  in  a 
CPA  less  than  500  ft.  Therefore,  of  all  the  30  degree, 
no-response  blunders  into  flight  simulators,  186  were  determined 
to  be  "at  risk." 

3.3.2  Sample  Size. 

The  TWG  had  previously  determined  that  controller  blunder 
resolution  performance  should  be  evaluated  against  a  baseline  of 
200  "at  risk"  blunders.  In  the  DIA  simulation,  200  "at  risk" 
blunders  could  not  be  accomplished  due  to  limited  simulation 
time.  Therefore,  the  method  of  sequential  sampling  was  u^'^d  to 
determine  an  acceptable  sample  size  for  the  DIA  simulatio.  <  that 
would  yield  the  same  net  effect  as  a  sample  of  200  blunders  and 
could  be  collected  within  time  constraints. 

This  sequential  sampling  method  was  based  on  the  assumption  that 
a  2  percent  TCV  rate  was  acceptable  for  a  sample  size  of  200  "at 
risk"  blunders  (e.g.,  with  200  "at  risk"  blunders,  4  TCV's  would 
be  acceptable) .  This  technique  was  designed  to  result  in  a  99 
percent  confidence  about  the  target  TCV  ratio  (i.e.,  2  percent 
TCV  rate) .  Thus,  if  a  test  ran  with  0  TCV's  out  of  86  "at  risk" 
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blunders,  then  the  upper  limit  of  the  .99  confidence  interval 
would  agree  with  the  results  obtained  from  a  test  with  4  observed 
TCV's  out  of  200  "at  risk”  blunders. 

Similar  calculations  were  made  assuming  1,  2,  3,  or  4  observed 
TCV's.  Table  3  presents  the  required  number  of  "at  risk" 
blunders,  assuming  0  to  4  TCV's  were  observed  during  the  DIA 
simulation. 

In  summary,  the  no-controller  intervention  runs  were  used  as  an 
estimate  of  the  niunber  of  "at  risk"  blunders  in  the  planning  and 
execution  of  the  simulation.  This  estimate  was  used  in  deriving 
the  number  of  blunders  that  had  to  be  initiated  to  yield  an 
appropriate  sample  of  "at  risk"  blunders.  The  aircraft  position 
data  were  used,  after  the  simulation,  to  validate  the  number  of 
blunders  "at  risk"  and  to  confirm  that  an  adequate  number  of 
blunders  had  been  initiated. 


TABLE  3.  REQUIRED  SAMPLE  SIZE 


Observed  TCV's 

Required  "At  Risk"  Blunders 

0 

86 

1 

119 

2 

151 

3 

179 

4 

200  1 

4^ _ SIMULATION  RESULTS. 

This  section  describes  analyses  of  the  data  collected  in  the  DIA 
simulation.  Examination  of  CPA's,  TCV's,  NTZ  entries,  and  NBO's 
is  presented.  CPA  data  are  initially  presented  for  all  blunder 
angles  and  simulator  types;  however,  the  focus  of  all  subsequent 
CPA  analyses  is  for  30  degree,  no-response  blunders  into  flight 
simulators.  Lastly,  a  risk  assessment  of  the  DIA  simulation  is 
detailed. 

Prior  to  the  simulation,  the  MPAP  TWG  determined  that  blunders 
with  TGF  aircraft  evaders  would  not  be  included  in  the  simulation 
analyses.  This  decision  was  due  to  differences  between  TGF 
aircraft  and  flight  simulators,  such  as;  1)  aerodynamic 
performance  between  TGF  aircraft  and  flight  simulators  (these 
differences  limited  the  generalizability  of  TGF  "pilot"  and 
"aircraft"  performance  to  the  operational  environment) ;  2)  the 
TGF  interface  for  the  SPO's  was  not  representative  of  flight 
simulator  controls  and  displays;  and  3)  SPO's  were  not  trained 
pilots,  they  were  keyboard  operators.  Therefore,  the  simulation 


21 


analyses  and  the  TWG  operational  assessment  were  based  on  data 
from  blunders  into  flight  simulators. 

4.1  OVERVIEW  OF  AIR  TRAFFIC. 

The  air  traffic  consisted  of  11,787  approaches  over  63  runs. 
Flight  simulators  made  1639  approaches,  and  TGF  aircraft  made 
10,148  approaches. 

_ Aircraft  Speeds. 

The  speeds  of  both  the  blundering  and  evading  aircraft  are 
critical  factors  in  the  successful  resolution  of  a  blunder. 
Aircraft  TAS  is  directly  related  to  M.S.L.  altitude  (see  table 
1) .  Groundspeed  is  a  function  of  TAS  and  wind.  Since  aircraft 
speeds  were  tantamount  to  the  integrity  of  the  simulation,  they 
were  examined  for  operational  validity.  Average  groundspeeds  of 
blundering  and  evading  aircraft  at  the  time  of  blunder 
initiation,  from  5000  ft  M.S.L.  to  11,000  ft  M.S.L. ,  were 
examined.  Speeds  ranged  from  140  to  180  IAS  until  the  aircraft 
slowed  to  final  approach  speeds.  These  data  indicated  that 
flight  simulator  and  TGF  aircraft  groundspeeds  were  consistent 
with  the  operational  environment. 

At  It. 2 _ Blunders. 

Approximately  14  blunders  were  initiated  per  hour.  Blunders 
were  10  degrees  in  54  blunders  (3.8  percent),  20  degrees  in  244 
blunders  (17.1  percent),  and  30  degrees  in  1132  blunders  (79.1 
percent).  There  was  no  response  in  999  blunders  (69.9  percent), 
and  there  was  a  response  in  431  blunders  (30.1  percent). 
Blundering  and  evading  aircraft  were  in  conflict  if  they  came 
within  3  nmi  horizontally  and  1000  ft  vertically.  Table  4 
presents  the  blunder  degree,  simulator  type,  and  response  data 
for  the  1430  blunders  that  resulted  in  a  conflict. 

4.1.3 _ Evading  Aircraft. 

The  evading  aircraft  was  a  flight  simulator  in  1170  of  all  the 
blunders  (81.8  percent).  Previous  simulations  indicated  that 
flight  simulator  evasive  maneuvers  were  executed  with  greater 
fidelity  than  those  initiated  by  TGF  aircraft.  Therefore,  cases 
with  flight  simulator  evading  aircraft  were  most  like  the 
operational  environment.  Thus,  only  data  from  blunders  into 
flight  simulators  were  analyzed  and  presented  in  the  results. 
Evading  flight  simulator  data  are  presented  in  table  5. 

4.2  DIA  SIMULATION  ANALYSES. 

The  following  section  examines  the  simulation  data  for  conflicts, 
"at  risk"  aircraft,  and  participant  opinions.  Analyses  were 
conducted  on  the  blunder  resolution  performance,  NBO  frequency. 


22 


and  NTZ  entry  frequency.  A  detailed  operational  assessment  of 
each  TCV  was  conducted  by  the  TWG.  Controller  post-run  and 
post-simulation  questionnaire  responses  were  qualitatively 
analyzed.  These  findings,  as  well  as  the  risk  assessment 
findings,  are  presented  in  the  following  sections. 

TABLE  4.  BLUNDERS  THAT  RESULTED  IN  A  CONFLICT 


Evading  Aircraft/Blunder  Degree 

Flight  Simulator  TGF  Aircraft 

(n  =  1170)  (n  =  260) 


10  20  30  10  20  30 


Response 
n  *  431 

0.8% 

n=12 

5.7% 

n=82 

16.5% 

n=236 

0.8% 

n=12 

1.5% 

n=22 

4.7% 

n*67 

No 

Response 

1  n  =  999 

1.4% 

n=20 

8.0% 

n=115 

49.3% 

n=705 

0.7% 

n=10 

1.7% 

n=25 

8.7% 

n=124 

TABLE  5.  EVADING  FLIGHT  SIMULATOR  DATA 


Flight  Simulator 

Number  of 
Conflicts 

Percent  of 
Conflicts 

MD-88 

207 

17.7 

B-727 

457 

39.0 

B-737 

191 

16.4 

B-747-400 

79 

6.8 

Lion 

108 

9.2 

GAT 

128 

4.2.1  Conflicts. 

Flight  simulators  were  the  evading  aircraft  in  1170  conflicts. 
The  full  range  of  CPA's  for  flight  simulator  evaders  is  shown  in 
figure  4.  The  mean  CPA  for  these  conflicts  was  3697  ft  (s.d.  = 
1484  ft,  range  280  ft  to  10,357  ft).  Table  6  presents  CPA  data 
for  flight  simulator  evaders. 
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FIGURE  4.  FULL  RANGE  OF  CPA*S,  FLIGHT  SIMULATOR  EVADERS 


TABLE  6 


EVADING  FLIGHT  SIMULATOR  CPA  DATA 


Runway  Pair/Blunder  Degree 

16/17R  (7600  ft)  17R/17L  (5280  ft) 

Resp 

10 

20 

30 

10 

20 

30 

Yes 

Avg.= 

6787 

5763 

4843 

4390 

3989 

3196 

s.d.- 

1226 

663 

1107 

1487 

776 

991 

n  = 

6 

32 

82 

6 

50 

154 

No 

Avg.= 

4883 

4865 

4295 

3460 

3269 

2748 

s.d.= 

1760 

1438 

1470 

990 

1422 

1129 

n  = 

13 

41 

292 

7 

4.2.2  Independent  Variables ♦ 

The  effects  of  blunder  angle  (10,  20,  or  30  degrees),  runway 
spacing  (5280  ft  or  7600  ft) ,  and  blundering  aircraft  response 
status  (on  or  off)  on  the  average  CPA  were  examined. 

As  expected,  30  degree  blunders  resulted  in  controllers 
maintaining  less  aircraft  separation  than  10  or  20  degree 
blunders.  Secondly,  blunders  that  did  not  respond  resulted  in 
controllers  maintaining  less  aircraft  separation.  Finally, 
blunders  initiated  between  the  runways  spaced  5280  ft  apart  (17R 
and  17L)  resulted  in  less  separation. 

As  expected,  the  CPA  data  confirmed  that  30  degree,  no-response 
blunders  were  a  worst-case  situation.  Therefore,  the  aircraft 
separation  data  from  30  degree,  no  response  blunders  into  flight 
simulators  were  examined  in  greater  detail.  The  distribution  of 
CPA's  for  these  blunders  is  presented  in  figure  5. 

Thirty  degree,  no-response  blunders  into  flight  simulators 
produced  conflicts  with  CPA's  that  ranged  from  280  ft  to  10,357 
ft.  The  mean  separation  for  these  blunders  was  3388.9  ft  (s.d.  = 
1490.5  ft) . 

4.2.3  CPA  Lower  Distribution  Analysis. 

Analyses  were  conducted  on  the  lower  tail  of  the  distribution 
resulting  from  30  degree,  no-response  blunders  into  flight 
simulators.  There  were  13  conflicts  in  which  the  CPA  came  within 
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INTO  FLIGHT  SIMULATORS 


1000  ft.  Table  7  presents  these  data.  Qualitative  analysis 
revealed  that  eight  conflicts  were  between  the  closer  spaced  17R 
and  17L  (i.e.,  5280  ft)  runways. 

4.2.4  _ Decision  Tree  Analysis. 

A  comprehensive  review  of  the  eight  conflicts  that  resulted  in  a 
miss  distance  of  less  than  500  ft  was  conducted.  This  review 
analyzed  video  tapes,  controller  response  times,  controller 
message  lengths,  pilot  response  times,  aircraft  performance,  site 
coordinator  notes,  technical  observer  logs,  and  controller 
incident  reports.  This  review  was  performed  to  ensure  that  each 
TCV,  a)  was  not  the  result  of  a  simulation  anomaly,  and  b)  could 
have  occurred  in  the  operational  environment. 

Three  of  the  eight  TCV's  had  TGF  aircraft  evader's.  The  review 
further  indicated  that  an  additional  three  TCV's  were  the  result 
of  simulation  anomalies.  In  one  case,  a  flight  simulator  target 
was  incorrectly  located.  In  the  second  case,  the  blunder  was 
initiated  less  than  2  nmi  from  the  runway  threshold,  which 
violated  procedures  outlined  in  the  test  plan.  In  the  third 
case,  the  blunder  was  initiated  after  the  run  was  completed. 

These  six  cases  were  removed  from  further  analysis. 

Two  TCV's  were  maintained  to  be  valid.  These  two  TCV's  are 
described  in  the  following  section. 

4.2.5  Test  Criterion  Violations. 

The  following  is  a  description  of  the  valid  TCV's  in  the  DIA 
simulation.  Both  TCV's  were  the  result  of  30-degree,  no-response 
blunders . 


4. 2. 5.1  Trans  World  Airlines  621  into  American  204. 

The  first  TCV  occurred  between  Trans  World  Airlines  621  (TWA621) 
and  American  204  (AAL204)  in  the  first  run  on  December  3,  1992 
(run  niunber  25)  .  Approximately  17  minutes  into  the  run,  TWA621 
on  17L  blundered  to  the  right  towards  AAL204,  a  flight  simulator 
on  17R.  The  ATC  message  to  the  endangered  aircraft  (AAL204) 
began  7  seconds  after  blunder  initiation. 

AAL204  acknowledged  and  responded  in  a  timely  manner.  The  14 
second  response  time  was  within  the  normal  range  of  aircraft 
response  times  (Hasman  F.  and  Pratt,  M. ,  1991;  Hasman,  F. ,  Pratt, 
M. ,  and  Jones,  A.,  1991).  The  lowest  horizontal  separation  was 
235  ft,  and  the  lowest  vertical  separation  was  188  ft.  The 
resultant  slant  range  miss  distance  was  301  ft.  Further  data, 
including  groundspeeds  and  altitudes  at  blunder  initiation,  can 
be  found  in  table  8. 
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TABLE  7.  CPA  LOWER  DISTRIBUTION  DATA 
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Note:  ALT  and  GS  at  blunder  initiation 

DIST  at  time  of  CPA. 


TABLE  8.  TCV  DATA,  TWA621/AAL204 


1  Blundering  Aircraft 

Evading  Aircraft 

Runway 

17L 

17R 

Call  Sign 

TWA621 

AAL204 

Groundspeed  at 
Blunder  Start  (kn) 

200 

184 

Altitude  at 

Blunder  Start  (ft) 

8307 

8457 

CPA  (ft) 

301  at  7.9  nmi  from 

runway  threshold 

The  pilot  of  the  evading  aircraft  was  flying  by  hand  (i.e.,  raw 
data)  at  the  time  of  the  blunder.  The  site  coordinator  log 
indicated  a  "normal”  breakout  maneuver  with  a  maximum  observed 
bank  angle  of  30  degrees.  The  site  coordinator  noted  that  the 
initial  ATC  instruction  was  a  turn  to  270  and  a  climb  to  11,000. 

A  second  heading  of  320  was  later  given. 

Table  9  is  a  record  of  the  dialogue  between  the  controllers  and 
pilots  during  the  conflict.  Transmission  times  were  extracted 
from  the  video  tape  of  the  run  and  reflect  real-time  minutes  and 
seconds  of  the  simulation  clock. 

One  technical  observer  attributed  the  TCV  to  slow  pilot  and 
aircraft  response.  The  observer  wrote,  "TWA  621,  NORDO.  17R 
pilot  slow  to  turn." 

The  controller  for  the  17R  runway  (i.e.,  evading  aircraft  runway) 
wrote  that  the  blunder  was  self-identified.  He  noted  that  he  saw 
the  blundering  aircraft  right  of  the  17L  localizer,  but  not  in 
the  NTZ.  The  controller  stated  that  he  immediately  turned  and 
climbed  AAL204,  before  TWA621  entered  the  NTZ.  The  controller 
identified  pilot  action  and  a  slow  aircraft  turn  as  the  causal 
factors  in  the  conflict.  The  controller  also  wrote  that,  "two 
factors  contributed  to  the  conflict:  1)  speed  overtake  - 
blundering  aircraft  was  flying  20  kn  faster  than  the  evading 
aircraft,  and  2)  the  evading  aircraft  was  slow  in  turning  and 
climbing."  It  was  recommended  that  to  prevent  a  recurrence  of 
this  type  of  conflict,  the  evading  aircraft  should  use  a  faster 
turn  rate. 

The  controller  for  the  blundering  aircraft  runway  (i.e.,  17L) 
noted  that  he  self-identified  the  blunder  when  TWA621  deviated 
from  course.  The  controller  attempted  to  turn  TWA621,  but  he  did 
not  get  a  response.  The  controller  also  identified  pilot  action 
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and  slow  pilot  response  of  AAL204  as  the  causal  factor  in  the 
conflict.  He  wrote  that,  ''AAL204  seemed  slow  in  responding  and 
breaking  out.” 


TABLE  9.  DIALOGUE  FOR  TCV,  TWA621/AAL204 


Rwy 

Transmission 
Start  End 

17L 

Controller:  "TWA  Six-Two-One.  Turn  Left 

Immediately,  Heading  Zero-Niner-Zero, 

Climb  and  Maintain  One-One-Thousand." 

17:45 

17:53 

17R 

Controller:  "American  Two-Zero-Four. 

Turn  Right  Immediately,  Heading  Two- 
Seven-Zero,  Climb  and  Maintain  One-One- 
Thousand.  " 

17:47 

17:51 

17R 

Controller:  "Turn  to  Two-Seven-Zero. 

Climb  to  One-One-Thousand,  American  Two- 
Zero-Four.  " 

17:51 

17:53 

17R 

Pilot:  "Climbing  to  One-One-Thousand, 

American  Two-Zero-Four." 

17:54 

17:55 

17R 

Controller:  "Two-Zero-Four  Expedite  Turn 

and  Climb." 

17 : 56 

17:58 

17R 

Controller:  "Two-Zero-Four.  Turn  Right 

Heading  Three-Two-Zero." 

18:03 

18:05 

17L 

Pilot:  "Three-Two-Zero,  American  Two- 

Zero-Four. " 

18:06 

18:08 

4. 2. 5. 2  American  West  234  into  Northwest  893. 

The  second  TCV  involved  American  West  (CACTUS)  234  (AWE234)  and 
Northwest  893  (NWA893) .  It  occurred  in  the  second  run  on 
December  8,  1992  (run  number  35).  Crosswinds  were  from  the  west 
for  flight  simulator  approaches.  Approximately  16  minutes  into 
the  run,  AWE234  on  runway  17R  turned  towards  NWA893,  a  flight 
simulator  on  runway  16.  Emair  243  (EME243) ,  a  TGF  aircraft,  was 
traveling  approximately  4  nmi  behind  NWA893  and  was  in  potential 
danger.  Information  pertaining  to  the  TCV,  including 
groundspeeds  and  altitudes  at  blunder  initiation,  can  be  found  in 
table  10. 

The  17R  controller  began  his  message  to  the  blundering  aircraft 
approximately  10  seconds  after  blunder  initiation.  This  response 
time  was  approximately  two  radar  updates.  Although  it  did  not 
affect  the  outcome  of  the  blunder  (the  blundering  aircraft  was 
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directed  not  to  respond) ,  the  17R  controller  mistakenly  called 
AWE234  "Air  Wisconsin  234."  Approximately  one  second  later,  the 
runway  16  controller  gave  an  evasive  instruction  to  NWA893. 
Without  waiting  for  acknowledgement  from  NWA893  (NWA893  never 
acknowledged  the  transmission) ,  the  runway  16  controller  gave 
EME243  an  evasive  instruction.  After  two  transmissions,  EME243 
acknowledged.  The  runway  16  controller  then  repeated  the  evasive 
instructions  for  NWA893.  NWA893  acknowledged  and  responded  in  a 
timely  manner.  When  NWA893  responded,  AWE234  had  already  come 
within  500  ft.  The  resultant  slant  range  miss  distance  was  280 
ft. 


TABLE  10.  TCV  DATA,  AWE234/NWA893 


1  Blundering  Aircraft  Evading  Aircraft  | 

Runway 

17R 

16 

Call  Sign 

AWE234 

NWA893 

Groundspeed  at 
Blunder  Start  (kn) 

194 

184 

Altitude  at 

Blunder  Start  (ft) 

8972 

8769 

CPA  (ft) 

280  at  10.3  nmi  from  runway 
threshold 

The  12  second  response  time  from  NWA893  was  within  the  normal 
range  of  aircraft  response  times,  as  demonstrated  by  two 
experiments  of  ATC  directed  missed  approaches  at  6  miles  from  the 
runway  threshold  (Hasman,  F.  and  Pratt,  M. ,  1991;  Hasman,  F., 
Pratt,  M. ,  and  Jones,  A.,  1991).  These  studies  indicated  that 
the  B-727  took  an  average  of  4.5  seconds  to  respond,  with  a  range 
from  2  seconds  to  16  seconds. 

Table  11  is  a  record  of  the  dialogue  between  the  controllers  and 
pilots  during  the  conflict.  Transmission  times  were  extracted 
from  the  video  tape  of  the  run  and  reflect  the  real-time  minutes 
and  seconds  of  the  simulation  clock. 

Comments  recorded  in  the  site  coordinator  log  from  NWA893  stated, 
"...the  controller  broke  us  out.  We  answered  and  broke  out 
immediately."  In  addition,  the  log  indicated  the  evading 
aircraft  pilot  was  using  the  flight  director  at  the  start  of  the 
blunder. 
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TABLE  11.  DIALOGUE  FOR  TCV,  AWE234/NWA893 


Rwy 

Transmission  I 
Start  End  | 

17R 

Controller:  "Air  Wisconsin  Two-Thirtv- 

Four,  Turn  Left  Immediately,  Heading  One- 
Seven-Zero,  Climb  and  Maintain  One-Zero- 
Thousand  .  ” 

11:55 

11:59 

16 

Controller:  "Northwest  Eiaht-Ninetv- 

Three,  Turn  Right  Immediately,  Heading 
Two-Seven-Zero,  Descend  Immediately, 
Maintain  Seven  Thousand.” 

11:57 

12:02 

16 

Contrckler:  "Emair  Two-Fortv-Three . 

Climb  Immediately,  Maintain  One-One- 
Thousand,  Turn  Right  Heading  Two-Seven- 
Zero." 

12:06 

12:10 

16 

Controller:  "Alriaht  Gentlemen  Listen 

Up.  Emair  Two-Forty-Three,  Right  Turn 
Two-Seven-Zero,  Up  to  Eleven.” 

12:14 

12:17 

16 

Pilot:  "Two-Seven-Zero.  Emair  Two-Fortv- 

Three . ” 

12:18 

12:20 

16 

Controller:  "Northwest  Eiaht-Ninetv- 

Three,  Right  Turn,  Two-Seventy, Descend  to 
Seven." 

12:20 

12:22 

16 

Pilot:  "Northwest  Eiaht-Ninetv-Three. 
Two-Seven-Zero,  Heading  Down  to  Seven 
Thousand . " 

12:23 

12:26 

One  technical  observer  attributed  the  TCV  to  slow  pilot  and 
aircraft  response.  It  was  indicated  that  ATC  instructions  were 
not  followed.  A  second  technical  observer  also  cited  slow  pilot 
response  as  the  cause  of  the  TCV.  The  observer  wrote,  ''AWE234 
NORDO.  17R  controller  recognized  blunder  immediately.  It 
appeared  that  there  was  a  comm  problem  with  16  pilot.” 

The  controller  for  runway  16  (i.e.,  evading  aircraft  runway) 
wrote  that  the  blunder  was  self-identified  with  the  aid  of  the 
alert  system.  To  prevent  a  recurrence  of  a  similar  operational 
error,  the  controller  suggested  that,  "pilots  be  trained  for 
listening  skills.” 

The  controller  for  the  blundering  aircraft  runway  (i.e.,  17R) 
cited  "numerous  slow  pilot  responses,”  and  commented  that  "NWA893 
on  RWY  16  was  extremely  slow  responding.” 


32 


4.2.6 _ Blunder  Resolution  Performance. 


Overall,  the  results  indicated  that  controllers  were  able  to 
maintain  the  test  criterion  miss  distance  of  500  ft  in  184  out  of 
186  "at  risk"  blunders.  Thus,  the  success  rate  was  98.9  percent 
for  all  "at  risk"  blunders.  In  the  DIA  simulation,  blunder 
resolution  performance,  when  evaluated  against  "at  risk" 
blunders,  exceeded  the  98  percent  success  rate  criterion  set  by 
the  TWG. 

_ AIRPORT  ARRIVAL  RATE. 

Arrival  rates  are  reduced  when  aircraft  are  broken  out  of  the 
approach  sequence.  Breakouts  from  the  approach  can  be  due  to 
several  causes,  such  as  blunders,  nuisance  breakouts,  and  NTZ 
entries.  Blunder- induced  breakouts  were  the  most  prevalent  in 
the  simulation.  Nuisance  breakouts  occurred  when  an  aircraft  was 
broken  out  of  its  final  approach  for  reasons  other  than  a 
conflict,  loss  of  longitudinal  separation,  or  lost  beacon  signal 
(i.e.,  aircraft  goes  into  coast).  NTZ  entries  could  also  cause 
breakouts.  Controllers  are  required  to  breakout  aircraft  in 
proximity  of  an  aircraft  that  enters  the  NTZ  (Federal  Aviation 
Administration,  1992) . 

Two  factors  that  contribute  to  NTZ  entries  and  nuisance  breakouts 
are:  1)  TNSE  and  2)  runway  spacing. 

_ TNSE. 

TNSE  is  the  difference  between  an  aircraft's  actual  and  intended 
paths.  TNSE  is  expressed  as  a  statistical  combination  of  all 
sources  of  navigation  error.  These  sources  include:  navigation 
signal  source,  propagation,  airborne  system,  and  flight  technical 
error. 

To  provide  adequate  vertical  separation  during  turn-on,  multiple 
parallel  approaches  require  localizer  intercepts  farther  from  the 
runway  threshold.  Farther  intercepts  increase  the  effects  of 
TNSE  due  to  dispersion  of  the  localizer  signal.  The  increase  in 
TNSE  generally  results  in  an  increase  in  NTZ  entries  and  NBO's. 

4.3.2  NTZ  Entry  Analysis. 

Analyses  were  planned  to  evaluate  NTZ  entries  that  were  not  the 
result  of  a  blunder  or  breakout.  Simulation  results  indicated 
that  there  were  no  NTZ  entries  by  either  flight  simulators  or  TGF 
aircraft. 

4.3.3  NBO  Analysis. 

NBO's  occurred  in  17  of  the  8927  non-blunder  approaches  (0.2 
percent) .  A  chi  square  goodness-of-fit  test  indicated  that  the 
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number  of  NBO's  was  nearly  level  across  all  runs  of  the 
simulation;  therefore,  there  did  not  appear  to  be  any  systematic 
bias  present  due  to  training  or  fatigue.  NBO  data  are  presented 
in  figure  6. 

The  data  also  indicated  that  5  out  of  the  17  (29.5  percent)  NBO's 
were  on  17L,  8  out  of  the  17  (47  percent)  were  on  17R,  and  4  out 
of  17  (23.5  percent)  were  on  runway  16.  Thus,  aircraft  that  flew 
the  more  closely  spaced  approaches  (i.e.,  17L  and  17R)  were 
broken  out  about  twice  as  frequently  as  aircraft  that  flew  the 
less  closely  spaced  16  approach.  In  addition,  most  aircraft 
broken  out  were  on  the  center  runway  (i.e.,  17R) .  This  runway 
had  an  increased  incidence  of  NBO's  due  to  the  two  adjacent 
approaches.  Of  all  the  NBO's  that  occurred,  3  out  of  17  (18 
percent)  of  the  broken  out  aircraft  were  flight  simulators.  The 
NBO  data  are  presented  in  table  12. 

4.4  CONTROLLER  QUESTIONNAIRE  ANALYSIS. 

The  following  sections  describe  the  results  of  the  analyses  on 
the  Post  Run  Controller  Questionnaires  and  the  Post  Simulation 
Controller  Questionnaires. 

4.4.1  Post  Run  Controller  Questionnaires. 

Questionnaires  were  distributed  to  monitor  controllers  after 
their  sessions  during  the  simulation.  Controllers  rated  their 
level  of  activity,  stress,  and  mental  effort  throughout  each  run. 

4. 4. 1.1  _ Activity  Level. 

Activity  level  was  rated  on  a  scale  ranging  from  1  (minimal)  to  5 
(intense) .  Of  270  completed  questionnaires,  98  percent  of  the 
responses  were  a  2  or  3.  This  indicated  a  minimal  to  moderate 
level  of  activity. 

A  one-way  analysis  of  variance  (ANOVA)  indicated  a  significant 
effect  (e  <  .05)  of  runway  assignment  (16,  17R,  17L)  on 
controller  activity.  A  Scheffe  post-hoc  pairwise  comparison  test 
showed  that  controllers  had  significantly  more  activity  (e  <  *05) 
on  runway  17R  (i.e.,  center  runway)  than  on  runway  16  (i.e.,  7600 
ft  spacing) .  A  higher  activity  level  was  expected  since  the 
center  runway  was  involved  in  each  blunder. 

4.4. 1.2  _ Stress  Level. 

Controllers  rated  their  level  of  stress  on  a  five  point  scale, 
ranging  from  1  (minimal)  to  5  (intense) .  The  results  paralleled 
the  activity  level  ratings  in  that  96  percent  of  the  responses 
were  minimal  or  moderate. 
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FIGURE  6 


TABLE  12.  NUISANCE  BRE/iKOUT  DATA 


Runway 

Simulator  Type 

#  of  Cases 

Percent 

16 

Flight  Simulator 

0 

0.0 

TGF 

4 

23.5 

17R 

Flight  Simulator 

1 

6.0 

TGF 

7 

41.0 

17L 

Flight  Simulator 

2 

12.0 

TGF 

3 

17.5 

Totals 

17 

100.0 

A  one-way  ANOVA  was  performed  to  detect  any  effects  of  runway 
assignment  on  stress  levels.  Runway  assignment  significantly 
affected  stress  level  (p  <  .05).  A  Scheffe  post-hoc  pairwise 
comparison  showed  that  controllers  had  significantly  more  stress 
(E  <  .05)  on  runway  17R  (i.e.,  center  runway)  than  on  runway  16 
(i.e.,  7600  ft  spacing).  This  relationship  was  expected  since 
the  center  runway  controller  was  involved  in  all  blunder  and 
breakout  situations. 

4t4tlt3 _ Mental  Effort. 

Mental  effort  was  rated  on  a  scale  ranging  from  1  (low)  to  5 
(maximum).  Overall,  64  percent  of  the  responses  were  twos  (i.e., 
"acceptable") .  Twenty- four  percent  of  the  responses  were  threes 
(i.e.,  "moderately  high").  These  percentages  varied  little  over 
the  course  of  the  simulation.  There  were  39  ratings  of  fours  and 
fives.  Of  these  ratings,  36  (92.3  percent)  were  made  by  1 
controller.  Thus,  one  controller  consistently  rated  a  high  level 
of  mental  effort,  indicating  a  rating  bias  toward  the  high  end  of 
the  scale. 

A  one-way  ANOVA  was  performed  to  determine  if  runway  assignment 
(16,  17R,  17L)  affected  the  level  of  mental  effort  to  maintain 
traffic  separation.  No  significant  difference  was  found  (p  > 
.05).  Thus,  it  appears  that  the  mental  effort  required  to  detect 
and  resolve  blunders  on  all  three  runways  was  the  same. 

Overall,  the  situational  factors  associated  with  runway 
assignment  (e.g.,  frequency  of  blunders,  frequency  of  breakouts) 
required  higher  activity  levels  and  produced  more  stress,  but  did 
not  require  any  additional  mental  effort.  Independent  of  runway 
assignment,  the  controllers'  mental  effort  was  generally 
moderate.  Further,  based  on  controller  blunder  resolution 
performance,  the  level  of  mental  effort  seemed  appropriate  for 
the  task. 
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Each  controller  was  given  a  post-simulation  questionnaire  at  the 
end  of  their  participation.  The  questions  addressed  the 
difficulty  of  the  controllers'  tasks  performed  during  the 
simulation.  Questions  also  asked  about  controller  blunder 
resolution  strategies  and  the  authenticity  of  the  simulation. 
Further,  controllers  were  queried  whether  they  believed  triple 
independent  instrument  flight  rules  (IFR)  approaches  to  runways 
spaced  5280  and  7600  ft  apart  could  be  safely  conducted  in  the 
operational  environment,  as  simulated.  Nine  out  of  the  ten 
controllers  completed  the  post-simulation  questionnaires. 

-4t4T2tl _ Task  Difficulty. 

The  scale  for  task  difficulty  using  the  FMA  ranged  from  1  (very 
difficult)  to  5  (very  easy) .  Controller  ratings  were  evenly 
distributed  from  3  (average  difficulty)  to  5  (very  easy) . 

4. 4. 2. 2 _ Specific  Strategies. 

The  most  common  "specific"  strategy  used  for  inter-controller 
coordination  was  a  verbal  warning  and/or  hand  signal. 

Controllers  also  told  each  other  the  instructions  that  were  given 
(e.g.,  descents,  climbs,  turns).  Often,  simply  hearing  another 
controller  transmit  instructions  acted  as  an  alert  and 
coordination  cue.  One  controller  developed  a  strategy  to  scan 
the  adjacent  localizer.  Three  controllers  maintained  they  did 
not  employ  a  specific  strategy. 

.4f.4T3,? _ Central  Strategies. 

Six  out  of  nine  controllers  claimed  that  no  central  strategies 
were  developed  due  to  the  high  density  altitude  environment.  One 
controller  asserted  that  when  an  evading  aircraft  was  at  a  low 
altitude,  it  was  best  to  keep  the  aircraft  low  and  turn  it  90 
degrees  off  the  localizer.  On  the  center  runway,  however,  this 
controller  claimed  it  was  better  to  climb  the  aircraft. 


Another  controller's  central  strategy  was  to  either  climb  or 
descend  aircraft  when  blunders  occurred  between  runways  16  and 
17R  (i.e.,  7600  ft  spacing).  Lastly,  one  controller  stated  that 
the  central  strategy  used  was  to  be  "cognizant  of  higher  speeds 
and  reduced  aircraft  performance." 

4 .4. 2. 4  High  Density  Altitude. 

Controllers  were  asked  to  comment  on  any  potential  effects  high 
density  altitude  had  on  the  runway  configuration,  as  simulated. 
Five  controllers  did  not  believe  high  density  altitude  was  a 
factor  that  affected  their  work.  One  controller  reported  he 
would  be  uncomfortable  working  without  the  FMA's,  and  another 


controller  reported  faster  speeds  were  the  only  noticeeUsle 
differences  in  aircraft  perfomance.  Two  controllers  did  not 
answer  the  question. 

4t4t3t? _ Simulation  Reality. 

The  scale  was  from  1  (Not  Realistic)  to  5  (Very  Realistic) . 

There  was  one  "1"  rating,  one  “2'*  rating,  one  "3"  rating,  four 
"4"  ratings,  and  one  "S"  rating. 

The  questionnaire  also  addressed  the  authenticity  of  the  entire 
simulation  as  a  whole.  One  controller  commented  that  the 
simulation  was  as  realistic  as  possible.  Four  controllers 
believed  30  degree  blunders  and  the  number  of  NORDO  aircraft  were 
unrealistic.  Three  controllers  believed  the  groundspeeds  were 
too  fast.  Two  controllers  commented  on  the  poor  communication 
between  flight  simulators  and  the  controllers.  One  controller 
suggested  finding  a  way  to  make  flight  simulator  and  SPO 
communications  sound  more  alike  to  add  to  the  realism.  The 
controller  also  suggested  simplifying  the  menus  on  the  FMA 
displays. 

All  controllers  agreed  that  triple  independent  IFR  approaches  to 
runways  spaced  5280  and  7600  ft  apart  could  be  safely  conducted, 
as  simulated. 

4 1  g  PILOT  QVEgTIQNWMRS  MALYglS  ♦ 

Participating  flight  simulator  pilots  were  asked  to  complete  a 
survey  following  the  simulation.  The  survey  consisted  of  six 
statements  about  the  simulated  operations  and  procedures.  Pilots 
rated  the  first  four  statements  on  a  scale  ranging  from  1 
(strongly  agree)  to  5  (strongly  disagree) .  Pilots  also  provided 
general  comments  and  suggestions  in  response  to  two  statements. 

Sixty-two  flightcrew  opinion  surveys  were  completed.  The  average 
years  of  experience  for  all  of  the  participating  pilots  was  23 
years.  The  range  of  experience  was  from  6  to  42  years. 

4.5.1 _ Vertical  Separation. 

The  first  statement  read:  "In  the  event  that  one  (or  both) 
aircraft  overshoot  the  localizer,  1000  ft  of  vertical  separation 
would  provide  an  acceptable  safety  margin  provided  aircraft 
maintain  their  assigned  altitude  until  established  on  the 
localizer  course."  Eighty-seven  percent  of  the  pilot  responses 
were  "strongly  agree"  and  "agree"  (54  pilots) .  Four  pilots  were 
neutral,  three  pilots  disagreed,  and  one  pilot  strongly 
disagreed. 
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4t§t2 _ Phraseology. 


The  second  statement  read:  "To  emphasize  the  impozi:ance  of  a 
quick  response  from  the  threatened  aircraft,  special  phraseology 
should  be  used  for  the  break-out  maneuver."  Ninety- five  percent 
(59  pilots)  of  the  responses  were  "strongly  agree"  and  "agree." 
Two  pilots  were  neutral  on  the  issue,  and  one  pilot  disagreed. 

4.5.3  Communications. 

The  third  statement  had  two  parts.  The  first  part  read:  "An 
alternate  communication  frequency  would  be  useful."  Pilot 
responses  tended  not  to  be  extreme  on  this  issue.  Thirteen 
percent  (8  pilots)  strongly  agreed.  Twenty-eight  percent  of  the 
respr  es  were  "agree"  (17  pilots),  23  percent  were  "neutral" 

(14  ]  ts) ,  and  23  percent  were  "disagree"  (14  pilots).  Ten 
percei.^  (6  pilots)  strongly  disagreed.  Three  pilots  did  not 
respond  to  this  statement. 

The  second  part  of  the  statement  read:  "Breakout  instructions 
should  be  broadcast  simultaneously  on  the  ILS  frequency  (voice  on 
the  localizer)."  Twenty-five  percent  of  the  pilots  agreed  (15 
pilots)  and  36  percent  disagreed  (22  pilots)  with  this 
recommendation.  Ten  percent  (6  pilots)  strongly  agreed,  13 
percent  (8  pilots)  were  neutral  on  the  issue,  and  16  percent  (10 
pilots)  strongly  disagreed.  One  pilot  did  not  respond  to  the 
statement . 

4.5.4  General  Comments. 

The  survey  also  asked  pilots  to  provide  any  suggestions  on  types 
of  special  terminology/phraseology  and/or  instructions  that  would 
be  effective  in  initiating  evasive  maneuvers.  The  most  popular 
suggestion  was  controller  repetition  of  the  aircraft  call  sign, 
at  the  beginning  or  the  end  of  the  instruction.  The  term 
"immediately"  was  acknowledged  by  more  pilots  as  the  effective 
term  for  instructing  an  escape  maneuver  rather  than  "escape." 

Another  common  suggestion  was  to  give  the  heading  and  altitude  in 
one  transmission,  with  the  most  urgent  instruction  first. 

Several  pilots  thought  there  should  be  no  "descending"  escapes. 
Several  pilots  favored  a  pviblished  escape  procedure  and  an 
established  standard  terminology. 

4.6  CONTROLLER  STATEMENT. 

In  their  controller  report,  the  participating  air  traffic 
controllers  concluded,  "In  this  simulation  the  air  traffic 
controller  team  believes  they  safely  monitored  triple 
simultaneous  ILS  approaches  at  the  simulated  new  Denver  Airport 
using  the  FMA." 
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4.7  TECHNICAL  OBSERVER  STATEMENT. 


Th«  technical  observers  unanimously  agreed  that  controllers 
appeared  to  separate  aircraft  with  relative  ease.  In  their 
report,  the  technical  observers  stated  that  during  the 
simulation,  the  controllers  had  little  difficulty  detecting  and 
resolving  blunders.  In  some  cases,  controller  response  times 
were  actually  slower,  as  the  controllers  took  time  to  evaluate 
the  situation  prior  to  issuing  instructions  to  the  evading 
aircraft.  Alert  algorithms  and  runway  spacings  appeared  to  be 
factors  which  allowed  controllers  to  take  extra  time  when 
evaluating  certain  situations. 

In  conclusion,  the  technical  observers  unanimously  agreed, 
"...triple  simultaneous  approaches  at  the  DIA  airport  can  be 
safely  accomplished  using  the  4.8  second  update  radar  and  the 
final  monitor  aid." 

4t9  RI$K  AgSEggHENT. 

The  analyses  above  have  assessed  the  controllers'  ability  to 
resolve  blunders  during  simultaneous  parallel  ILS  approaches.  It 
was  observed  that  the  controllers  were  not  always  able  to 
maintain  the  test  criterion  miss  distance  of  more  than  500  ft 
between  the  blundering  aircraft  and  aircraft  on  adjacent 
approaches.  To  properly  evaluate  the  proposed  operations,  the 
effect  of  implementing  the  proposed  operation  on  the  level  of 
risk  found  in  today's  operational  environment  must  be  determined. 

Ideally,  the  probability  of  an  accident  during  the  approach 
operation  could  be  computed  and  compared  to  an  acceptable 
probability.  However,  since  the  majority  of  approach  operations 
are  conducted  to  single  approaches,  no  recorded  accident  has  ever 
been  attributable  to  a  blunder  occurring  during  multiple  approach 
operations.  Accordingly,  little  effort  has  been  made  to  record 
and  track  blunder  occurrences. 

A  way  to  complete  the  risk  analysis  without  operational  blunder 
data  is  to  determine  a  target  risk  value,  and  then  to  compute  a 
blunder  rate  which  would  result  in  the  target  risk  value.  This 
would  provide  insight  to  the  safety  level  of  the  operation,  and 
allow  the  FAA  to  assess  the  acceptability  of  the  operation.  The 
computed  blunder  rate  should  not  be  used  as  the  sole  determining 
factor  of  whether  the  operation  meets  the  target  risk  value. 

The  total  nvunber  of  air  carrier  accidents  as  well  as  the  number 
of  fatal  accidents  on  final  approach  has  been  extracted  from  the 
National  Transportation  Safety  Board  (NTSB)  data  for  the  time 
period,  1983-1989.  This  number,  together  with  the  total  number 
of  ILS  approaches  during  the  time  period,  lead  to  an  estimated 
fatal  accident  rate  during  ILS  operations  performed  during  IMC  of 
4  X  10-’^  fatal  accidents  (ACC)  per  approach  (APP) .  There  are  a 
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number  of  causes  of  accidents  during  final  approach,  such  as 
structural  failure,  engine  failure,  or  midair  collision.  An 
initial  estimate  is  that  there  are  nine  possible  causes  of 
accidents  on  final  approach.  A  tenth  possible  accident  cause,  a 
collision  with  aircraft  on  an  adjacent  approach,  is  created  with 
the  implementation  of  parallel  approaches. 

For  simplicity  of  model  development,  it  is  assumed  that  the  risks 
of  the  10  potential  accident  causes  are  approximately  equal.  Thus 
the  contribution  of  any  one  of  the  accident  causes  would  be 
approximately  one-tenth  of  the  total  accident  rate.  Therefore, 
the  target  safety  level  for  midair  collisions  on  simultaneous 
parallel  approaches  is  4  x  10-8,  or: 

1  ACC _ 

25  mill  APP 

To  begin  the  evaluation,  CPA  analyses  indicated  that  controllers 
had  the  greatest  difficulty  in  maintaining  a  500  ft  spacing 
between  aircraft  in  the  event  of  a  30  degree  blunder.  Twenty 
(20)  and  10  degree  blunders  were  all  resolvable.  The  simulation 
also  demonstrated  that  only  blunders  which  simulated  a  lack  of 
response  by  the  blundering  aircraft  were  sometimes  unresol vable. 
The  pilot's  inability  to  respond  may  be  due  to  a  conflict  with 
another  radio  transmission,  weather  conditions,  or  a  malfunction 
of  the  aircraft.  Other  studies  (Precision  Runway  Monitor  Program 
Office,  1991)  have  estimated  that  only  one  percent  of  the 
aircraft  blundering  30  degrees  off  course  would  be  unable  to 
respond  to  controller  commands. 

It  is  assumed  that  pilots  will  be  able  to  resolve  conflicts 
during  visual  flight  rules  (VFR)  conditions,  therefore  only  IMC 
conditions  are  used  in  this  analysis.  Based  upon  these  findings 
and  assumptions,  a  worst  case  blunder  (WCB)  is  defined  as  a  30 
degree  blunder,  under  IMC  conditions,  in  which  the  blundering 
aircraft's  pilot  is  unable  to  respond  to  the  controller's 
directions  and  enters  the  NTZ. 

A  factor  needed  in  the  risk  assessment  is  the  probability  of  a  30 
degree  blunder  in  which  the  pilot  of  the  blundering  aircraft  was 
unable  to  comply  with  ATC  instructions  (i.e.,  a  WCB).  As 
mentioned  earlier,  previous  research  estimated  that  the 
probability  of  a  no-response  blunder  was  1/100.  Therefore,  the 
ratio  of  WCB's  to  30  degree  blunders  is: 

_ 1  WCP _ 

100  30-Degree  Blunder 

The  longitudinal  alignment  of  the  aircraft,  relative  to  the 
threshold,  on  adjacent  ILS  approaches,  was  found  to  be  an 
important  factor  in  conflict  resolution.  The  probability  of  a 


41 


blunder  resulting  in  a  TCV  Is  highest  when  the  blundering 
aircraft  is  slightly  ahead  (closer  to  the  threshold)  of  the 
adjacent  aircraft. 

For  analytical  purposes,  an  alignment  "window"  will  be  defined 
for  blunders  which  would  result  in  a  TCV  if  the  controllers  did 
not  intervene.  These  blunders  were  considered  to  be  "at  risk"  of 
resulting  in  a  TCV.  The  length  of  the  window  depends  on  the 
ratio  of  the  speeds  and  the  blunder  angle,  and  it  can  be  shown 
analytically  to  be  independent  of  the  runway  separation.  The 
speeds  used  in  the  simulation  ranged  from  120  kn  to  227  kn.  The 
120  kn  was  the  slowest  speed  of  a  blundering  aircraft,  and  the 
227  kn  was  the  fastest  speed  of  an  evading  aircraft. 

Assuming  that  either  aircraft  could  be  traveling  at  any  speed 
between  these  two  ntimbers,  then  the  ratio  of  speeds  ranged  from 
120/227  s  .53  to  227/120  -  1.89.  Using  the  maximum  speed  ratio 
(1.89)  with  a  blunder  angle  of  30  degrees,  the  maximum  window 
length  was  2279  ft.  Therefore,  the  probability  of  an  "at  risk" 
blunder,  assuming  3  nmi  longitudinal  separation  between  aircraft 
on  the  same  approach,  is  given  by: 

2279  =  .125  =  1  "at  risk"  WCB 

3  X  6076  8  WCB'S 

Therefore,  about  one  aligned  approach  occurs  for  every  eight 
approaches  executed. 

A  review  of  the  data  indicated  that  a  total  of  186  WCB's  were 
initiated  when  the  blundering  aircraft  was  in  the  alignment 
window.  Of  the  "at  risk"  WCB's,  two  resulted  in  an  actual  TCV. 
Using  a  99  percent  confidence  interval  to  compute  the  upper  bound 
for  the  probability  of  a  TCV  given  an  "at  risk"  WCB,  the  upper 
bound  would  be  0.049,  or: 

0.049  =  5  TCV^s _ 

102  "at  risk"  WCB's 

The  NTSB  would  evaluate  a  mid-air  collision  as  two  accidents. 
Therefore,  one  TCV  would  equal  two  accidents: 

1  TCV 

2  ACC 

Finally,  using  the  data  cited  above,  the  number  of  blunders  which 
could  occur  in  the  operational  environment,  before  the  target 
probability  of  1  fatal  accident  in  25  million  approaches,  can  be 
calculated.  The  calculation  is  shown  on  the  following  page: 
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25  mill  APP  1  "at  risk"  WCB 


5  TCV's 


1  Triple  APP 


X  1  TCV  X  100  30-Deqree  BL  =  9.8  30 -Degree  Blunder 

2  ACC  1  WCB  10,000  Triple  APP 

Therefore,  about  ten  30-degree  blunders  per  10,000  triple 
parallel  simultaneous  approaches  could  be  tolerated  for  the  risk 
of  the  operation  to  meet  the  target  risk  level. 

The  occurrence  of  blunders  during  parallel  approaches  has 
remained  undocumented.  Anecdotal  evidence  has  indicated  that 
blunders  do  occur  during  simultaneous  approach  operations. 
Knowledgeable  representatives  from  the  FAA  have  indicated  that 
blunders  may  occur  as  often  as  one  or  two  times  per  10,000 
simultaneous  approaches.  Therefore,  based  upon  the  data 
collected  in  the  simulation,  the  proposed  triple  parallel 
approach  operation  at  OIA  meets  the  current  high  level  of  safety 
found  in  approach  operations.  Additional  detail  about  the  risk 
analysis  of  blunders  appears  in  appendix  L. 

4.9  MPAP  TWG  STATEMENT. 

In  their  Operational  Assessment  (appendix  K) ,  the  MPAP  TWG 
stated,  "Based  on  the  established  test  criteria,  the  controllers 
met  the  simulation  objective.  The  arrival  monitor  positions  in 
the  simulation  proved  to  be  operationally  effective  and  feasible. 
The  test  controllers  participated  in  the  simulation  as  though 
they  were  controlling  live  traffic.  Their  attention  and 
dedication  was  critical  to  the  success  of  the  simulation. 

Based  upon  the  results  of  the  simulation,  the  TWG  believes  that 
the  proposed  triple  simultaneous  ILS  approaches  at  DIA  are 
acceptable,  achievable,  and  safe  with  the  final  monitor  aid  (FHA) 
system  and  an  appropriate  radar  system,  such  as  a  Mode  S 
monopulse  system  or  an  ASR-9  radar  system  enhanced  to  provide 
improved  target  resolution  capabilities." 

5.  DISCUSSION. 


This  discussion  covers  the  simulation  findings  with  respect  to 
the  criteria  set  for  the  DIA  simulation:  1)  the  number  of  TCV's 
relative  to  the  total  number  of  "at  risk"  blunders;  2)  frequency 
of  NTZ  entries  and  NBO's;  3)  operational  assessment;  and  4)  risk 
assessment . 

5.1  TEST  CRITERION  VIOLATIONS. 

During  the  simulation,  only  two  "at  risk"  blunders  resulted  in 
TCV's.  Since  there  were  186  "at  risk"  blunders  generated  during 
the  simulation,  the  controllers  were  able  to  successfully  resolve 
98.9  percent  of  all  "at  risk"  blunders.  This  98.9  percent 


blunder  resolution  percentage  exceeded  the  98  percent  criteria 
set  by  the  TWG.  Thus,  the  controller  blunder  resolution 
performance  was  satisfactory  in  the  DIA  simulation. 

5.2  NBO^S  AND  NTZ  ENTRIES. 

An  examination  was  conducted  for  NTZ  entries  that  were  not  the 
result  of  a  blunder  or  a  breakout.  Simulation  data  revealed  that 
there  were  no  NTZ  entries  by  flight  simulators  or  TGF  aircraft. 
Therefore,  the  DIA  runway  spacings  were  sufficiently  large  that 
aircraft  did  not  enter  the  NTZ  as  a  consequence  of  TNSE. 

NBO's  were  typically  the  result  of  TNSE.  NBO's  occurred 
infrequently  in  the  DIA  simulation.  Data  indicated  that  0.2 
percent  of  all  non-blundering  aircraft  were  broken  out  for 
reasons  other  than  a  conflict,  loss  of  longitudinal  separation, 
or  loss  of  beacon  signal  (l.e.,  aircraft  goes  into  coast). 
Overall,  the  low  number  of  NBO's  in  the  simulation  indicated  that 
controllers  could  accurately  assess  potential  blunder  situations. 

5.3  OPERATIONAL  ASSESSMENT. 

The  operational  assessment  of  the  DIA  simulation  was  based  on  the 
opinions,  conclusions,  and  recommendations  of  the  participating 
controllers,  technical  observers,  and  the  TWG. 

_ Controller  Assessment. 

Controllers  were  asked  to  rate  their  stress  level,  activity 
level,  and  mental  effort.  Controllers  rated  their  activity  and 
stress  levels  as  minimal  to  moderate.  There  was  more  stress  and 
activity  for  controllers  working  the  center  runway.  This  was 
expected  since  the  center  runway  was  involved  in  all  blunder  and 
breakout  situations. 

Controllers  rated  their  mental  workload  as  acceptable  to  moderate 
throughout  the  simulation.  The  amount  of  mental  effort  was  not 
related  to  runway  assignment.  Situational  factors  associated 
with  runway  assignment  (e.g.  frequency  of  blunders,  frequency  of 
breakouts)  required  higher  activity  levels  and  produced  more 
stress,  but  did  not  require  any  additional  mental  effort. 
Throughout  the  simulation,  based  on  blunder  resolution 
performance,  the  level  of  mental  effort  seemed  appropriate  for 
the  task. 

The  participating  controllers  agreed  that  they  "safely  monitored 
triple  simultaneous  ILS  approaches  at  the  simulated  new  Denver 
Airport  using  the  FMA." 
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5^3.2 _ Technical  Observer  Assessment. 


The  technical  observers  concluded  that  the  triple  approach 
operation  at  DIA  could  be  conducted  safely,  based  upon  their 
observations  of  controller  and  system  performance.  The  technical 
observers  reported  that  controllers  had  little  difficulty 
detecting  and  resolving  blunders  for  this  operation.  It  was 
noted  that  the  controllers  used  the  FMA  alerts  to  their 
advantage,  and  often  used  the  additional  lead  time  in  determining 
the  optimal  evasion  maneuver. 

$t3T3 _ TWG  Assessment. 

Based  on  the  established  test  criteria,  the  TWG  concluded  that 
the  controllers  met  the  simulation  objective.  The  arrival 
monitor  positions  in  the  simulation  proved  to  be  operationally 
effective  and  feasible. 

Based  on  the  results  of  the  simulation,  the  TWG  concluded  in  its 
operational  assessment  (appendix  K)  that  the  proposed  triple 
simultaneous  ILS  approaches  at  DIA  are  acceptable,  achievable, 
and  safe  with  the  FMA  system  and  an  appropriate  radar  system, 
such  as  a  Mode  S  monopulse  system  or  an  ASR-9  system  enhanced  to 
provide  improved  target  resolution  capabilities. 

Based  on  their  operational  assessment,  the  MPAP  TWG  made  three 
recommendations:  1)  there  should  be  one  monitor  controller  for 
each  runway;  2)  monitor  positions  should  be  located  adjacent  to 
one  another;  3)  a  radar  system  with  2.7  milliradians  azimuth 
accuracy  or  better  and  the  capability  to  resolve  two  aircraft  at 
20  nmi  separated  by  0.9  degrees  or  more  should  be  used. 

5.4  RISK  ASSESSMENT. 

A  risk  assessment  was  conducted  on  the  data  from  the  simulation, 
since  there  is  no  recorded  operational  data  about  blunders,  this 
analysis  determined  a  target  risk  value,  and  then  computed  a 
blunder  rate  which  would  result  in  the  target  risk  value.  This 
assessment  was  based  on  NTSB  data  for  the  total  number  of  air 
carrier  accidents,  as  well  as  the  number  of  fatal  accidents  on 
final  approach.  A  risk  model  was  developed,  and  it  was 
determined  that  about  ten  30-degree  blunders  per  10,000  triple 
parallel  simultaneous  approaches  could  be  tolerated  for  the  risk 
of  the  operation  to  meet  the  target  risk  level.  Thus,  the  risk 
assessment  indicated  that  the  DIA  operation  meets  the  target  risk 
of  4  X  10*®  approaches. 

6 .  CONCLUSIONS . 

This  simulation  tested  the  controllers  ability  to  effectively 
resolve  conflicts  for  the  proposed  triple  simultaneous  instrximent 
landing  system  (ILS)  operation  at  the  new  Denver  International 
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Airport  (DIA) .  In  addition,  the  simulation  examined  the 
influence  of  the  runway  spacing,  5280  and  7600  feet  (ft),  and  the 
high  density  altitude,  5431  ft  mean  sea  level,  on  no 
transgression  zone  (NTZ)  entries  and  nuisance  breakouts  (NBO's) . 
Controllers  used  final  monitor  aid  displays  to  monitor  approach 
traffic.  The  simulated  radar  sensor  had  the  performance  of  an 
Airport  Surveillance  Radar  (ASR)->9  system  enhanced  to  provide 
Improved  target  resolution  capabilities.  The  Multiple  Parallel 
Approach  Program  Technical  Work  Group  (TWG)  evaluated  the 
controllers  effectiveness  at  resolving  conflicts,  the  frequency 
of  NTZ  entries  and  NBO's,  and  the  ability  of  the  system  to 
maintain  a  predetermined  target  level  of  risk  (1  fatal  accident 
per  25,000,000  approaches).  Based  upon  their  evaluations,  the 
TWG  concluded  that  the  triple  simultaneous  ILS  approach  operation 
at  DIA  was  acceptable. 
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GLOSSARY 


Airport  Surveillance  Radar  fASR^  -  Approach  control  radar  used  to 
detect  and  display  an  aircraft's  position  in  the  terminal  area. 
ASR  provides  range  and  azimuth  information  but  does  not  provide 
elevation  data.  Coverage  of  the  ASR  can  extend  up  to  60  miles. 

Automated  Radar  Terminal  System  fARTS^  IIIA  -  The  modular, 
prograiamable  ARTS  which  detects,  tracks,  and  predicts  primary  as 
well  as  secondary  radar-derived  aircraft  targets.  This 
sophisticated  computer  driven  system  upgrades  the  existing  ARTS 
by  providing  improved  tracking,  continuous  data  recording,  and 
failsoft  capabilities. 

Blunder  -  An  unexpected  turn  by  an  aircraft  already  established 
on  the  localizer  toward  another  aircraft. 

Closest  Point  of  Approach  fCPA^  -  The  smallest  slant  range 
distance  between  two  aircraft  involved  in  a  conflict.  The 
distance  is  measured  from  the  center  of  each  aircraft  involved. 

Chi  square  -  A  statistical  test  for  determining  goodness-of-f it. 
Goodness-of-fit  is  an  expression  of  how  well  any  set  of  observed 
data  conforms  to  some  expected  distribution.  Tests  for 
evaluating  goodness-of-fit  are  based  on  the  sum  of  squared 
deviations  between  the  observed  and  expected  values. 

Conflict  -  Occurs  whenever  two  or  more  aircraft  approach  each 
other  with  less  than  the  minimum  allowable  airspace  separation. 

A  conflict  occurs  if  there  is  less  than  a  minimvim  of  1000  ft 
vertical  and  a  minimum  distance  of  3  nautical  miles  (nmi)  between 
aircraft. 

CPA  Prediction  Tool  -  Presents  a  window  of  aircraft  alignments 
for  predicting  separation  between  aircraft. 

Final  Monitor  Aid  (FMA)  -  A  high  resolution  color  display  that  is 
equipped  with  the  controller  alert  system  hardware/software  which 
is  used  in  the  precision  runway  monitor  (PRM)  system.  The 
display  includes  alert  algorithms  providing  the  target 
predictors,  a  color  change  alert  when  a  target  penetrates  or  is 
predicted  to  penetrate  the  no  transgression  zone  (NTZ) ,  a  color 
change  alert  if  the  aircraft  transponder  becomes  inoperative, 
synthesized  voice  alerts,  digital  mapping,  and  like  features 
contained  in  the  PRM  system. 

Flight  Technical  Error  fFTEl  -  The  accuracy  with  which  the  pilot 
controls  the  aircraft  as  measured  by  the  indicated  aircraft 
position  with  respect  to  the  indicated  command  or  desired 
position.  It  does  not  include  procedural  blunders. 
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Glide  Slope  Intercept  fGSU  -  The  minimum  altitude  to  intercept 
the  glide  slope  during  a  precision  approach.  The  intersection  of 
the  published  intercept  altitude  with  the  glide  slope,  designated 
on  Government  charts  by  the  lightning  bolt  symbol,  is  the 
precision  Final  Approach  Fix  (FAF) ;  however,  when  ATC  directs  a 
lower  altitude,  the  resultant  lower  intercept  position  is  then 
the  FAF. 

Groundspeed  fGS)  -  The  actual  speed  of  an  airplane  over  the 
ground.  It  is  true  airspeed  adjusted  for  the  wind.  A  headwind 
decreases  GS  while  a  tailwind  increases  it. 

Indicated  Airspeed  flAS)  -  The  reading  taken  directly  from  the 
airspeed  indicator  on  an  airplane.  IAS  does  not  reflect 
variations  in  air  density  as  higher  altitudes  are  reached. 

Instrument  Meteorological  Conditions  flMO  -  Meteorological 
conditions  expressed  in  terms  of  visibility,  distance  from  cloud, 
and  ceiling  less  than  the  minima  specified  for  visual 
meteorological  conditions . 

Missed  Approach  -  A  maneuver  conducted  by  a  pilot  when  an 
instrument  approach  cannot  be  completed  to  a  landing.  The  route 
of  flight  and  altitude  are  shown  on  Instrument  approach  procedure 
charts.  A  pilot  executing  a  missed  approach  prior  to  the  Missed 
Approach  Point  (MAP)  must  continue  along  the  final  approach  to 
the  MAP.  The  pilot  may  climb  immediately  to  the  altitude 
specified  in  the  missed  approach  procedure. 

Monitor  Controller  -  A  controller  who  continuously  monitors 
aircraft  conducting  parallel  instrument  landing  system  (ILS) 
approaches . 

National  Airspace  System  fNAS)  -  The  common  network  of  U.S. 
airspace;  air  navigation  facilities,  equipment  and  services, 
airports  or  landing  areas;  aeronautical  charts,  information  and 
services;  rules,  regulations  and  procedures,  technical 
information  and  manpower  and  material.  Included  are  system 
components  shared  jointly  with  the  military. 

No  Transgression  Zone  fNTZ)  -  A  critical  2000  ft  wide  zone 
between  parallel  runways  where  aircraft  are  prohibited  from 
entering.  It  is  established  equidistant  between  extended  runway 
centerlines. 

Nuisance  Breakout  fNBO)  -  Occurs  when  an  aircraft  is  broken  out 
of  its  final  approach  for  reasons  other  than  a  blunder,  loss  of 
longitudinal  separation,  or  lost  beacon  signal  (i.e.,  aircraft 
goes  into  coast) . 
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Outer  Marker  -  A  marker  beacon  at  or  near  the  glide  slope 

Intercept  altitude  of  an  ILS  approach.  It  is  keyed  to  transmit 
two  dashes  per  second  on  a  400  Hertz  (Hz)  tone,  which  is  received 
aurally  and  visually  by  compatible  airborne  equipment.  The  OM  is 
normally  located  4  to  7  miles  from  the  runway  threshold  on  the 
extended  centerline  of  the  runway. 

Parallel  Instrument  Landing  System  fILS)  Approaches  -  Approaches 
to  parallel  runways  by  aircraft  flying  under  Instrument  Flight 
Rules  (IFR)  which,  when  established  inbound  toward  the  airport  on 
the  adjacent  final  approach  courses,  are  radar-separated  by  at 
least  2  miles. 

Scheffe  Test  -  A  statistical  procedure  for  determining  which 
specific  comparisons  in  a  multi-factor  experiment  are 
significant.  These  comparisons,  which  are  run  after  the 
experiment  is  complete,  are  used  only  when  a  preliminary  analysis 
has  shown  overall  significance. 

Simulation  Pilot  Operator  rSPO)  -  A  person  who  operates  a  Target 
Generation  Facility  (TGF)  position  and  controls  the  trajectory  of 
TGF  aircraft  by  computer  input  messages.  The  SPO  will  usually 
communicate  via  voice  circuits  to  ATC  controller  personnel  in  the 
laboratory  which  is  being  used  to  simulate  an  operational 
facility. 

Simultaneous  Instrument  Landing  System  (ILS^  Approaches  -  An 
approach  system  permitting  simultaneous  ILS  approaches  to 
airports  having  parallel  runways  separated  by  at  least  4,300  feet 
between  centerlines.  Integral  parts  of  the  total  system  are  ILS, 
radar,  communications,  ATC  procedures,  and  appropriate  airborne 
equipment. 

Target  Generation  Facility  (TGF)  -  An  advanced  simulation  system 
designed  to  support  testing  of  current  and  future  ATC  systems  at 
the  FAA  Technical  Center.  The  TGF  is  capable  of  modeling  a 
logical  view  of  the  ATC  environment  (airspace  volume  including 
geographic  data,  weather  data,  navigation  aids,  radar  sensors, 
airport  data,  and  air  routes)  as  well  as  simulate  dynamic  data 
associated  with  the  movement  and  control  of  aircraft  through  the 
selected  airspace. 

Target  Generation  Facility  (TGF)  Aircraft  -  Targets  generated  by 
the  TGF  at  the  FAA  Technical  Center.  TGF  aircraft  were  used  to 
provide  additional  traffic  and  to  initiate  blunders. 

Technical  Observer  -  An  individual  who  monitors  each  control 
position  visually  and  aurally  during  each  simulation  run.  Their 
duties  include:  documenting  discrepancies  between  issued  control 
instzxictions  and  actual  aircraft  responses;  assisting  in  alerting 
responsible  parties  to  correct  any  problems  which  may  occur 
during  the  test  (e.g.,  computer  failure,  stuck  microphone); 
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assisting  controllers  in  preparation  of  reports;  and  assisting  in 
final  evaluation  of  data  in  order  to  prepare  a  Technical  Observer 
report  at  the  end  of  the  sinulation. 

Test  Criterion  Violation  fTCV)  -  Occur  when  two  aircraft  come 
within  500  ft  of  one  another  after  the  initiation  of  a  blunder. 

Test  Director  -  Individual  responsible  for  determining  the 
occurrence  of  bl\inders  through  the  use  of  the  Closest  Point  of 
Approach  (CPA)  Prediction  Tool  and  by  assessing  the  blunder 
scripts.  The  test  director  is  the  liaison  between  the  FAA 
Technical  Center  and  the  flight  simulator  sites  during  the 
simulation.  The  test  director  also  coordinates  the  response 
condition  with  the  target  generation  facility  (TGF)  aircraft 
operators . 

Total  Navigation  System  Error  fTNSE^  -  Represents  the  difference 
between  the  actual  fllghtpath  of  the  aircraft  and  the  path  it  is 
intending  to  fly.  It  is  caused  by  flight  techlncal  error  (FTE) , 
avionics  error,  instrument  landing  system  (ILS)  signal  error, 
weather,  and  pilot  performance. 

True  Airspeed  fTAS^  -  The  true  speed  of  an  airplane  through  the 
air.  As  altitude  or  air  temperature  increase,  the  density  of  the 
air  decreases.  For  a  given  indicated  airspeed  (IAS),  this  means 
the  TAS  Increases  with  altitude. 
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4.8  s 
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Field 
Elevation 
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Field 
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APPENDIX  B 

SIMULATION  USING  THE  FDADS 
CONTROLLER  REPORT 


ASSESSMENT  OF  TRIPLE  SIMULTANEOUS  PARALLEL  ILS  APPROACHES  AT 
THE  NEW  DENVER  INTERNATIONAL  AIRPORT  USING  THE  FDADS  AND  4.8 
SECOND  RADAR  UPDATE  RATE. 


Bargaining  unit  controllers  from  the  Denver  TRACON  and  airports  around  the  country 
were  asked  to  participate  in  a  real-time  air  traffic  control  simulation.  This  simulations 
was  scheduled  from  November  16, 1992  to  December  15,  1992.  The  controllers  involved 
in  this  study  were  tasked  to  provide  air  traffic  control  separation  service  for  a  high 
density  altitude  airport.  The  airport  configuration  is  similar  to  the  new  Denver  Airport. 
Field  elevation  in  this  simulation  was  5431  feet.  The  radar  monitoring  equipment  used  a 
4.8  second  update  rate  source  displayed  on  the  FDADS.  Aircraft  used  in  this  simulation 
were  provided  by  five  flight  simulators,  the  FAA  Technical  Center’s  general  aviation 
trainer,  and  the  target  generation  facility.  Aircraft  flew  approach  speeds  of  170  knots  for 
jet  and  turboprops  and  150  knots  for  light  twin  aircraft. 

This  simulation  evaluated  the  FDADS  equipment  during  the  first  two  days  of  the 
simulation.  The  remaining  three  weeks  of  the  simulation  was  directed  by  upper 
management  levels  to  evaluate  the  Final  Monitor  Air  (FMA)  as  the  display  tool. 

The  controllers  cited  the  system’s  effectiveness  to  display  information  as  a  causal  factor 
for  inadequate  blunder  detection.  The  controller’s  comments  best  illustrate  this 
statement.  "Trend  information  makes  it  difficult  to  discern  blunders."  "Slow  target 
update.  There  are  times  with  a  blunder  of  thirty  degrees  and  the  blundering  aircraft 
goes  NORDO  as  in  this  case,  not  enough  time  to  escape."  "Slow  target  update,  not 
enough  time  to  see  the  blunder  and  escape."  "Slow  target  update,  this  hinders  the 
controller  to  see  the  blunder  in  time  to  correct  the  situation."  "Equipment  is  unable  to 
display  information  accurately  enough  for  controllers  to  do  their  job." 

In  many  of  the  blunders,  the  controllers  also  listed  pilot  response  as  a  causal  factor  for  a 
blunder.  In  one  instance  a  controller  reported  that  after  the  pilot  acknowledged  the 
breakout  instruction,  the  aircraft  traveled  1  1/2-2  miles  before  the  aircraft  was  observed 
to  turn  away  from  the  final  approach  course.  However,  another  controller’s  description 
of  his  observation  in  a  three  airplane  escape  indicates  the  difference  in  pilot/aircraft 
systems  reactions  times.  The  controller  wrote,  "AAL745  did  an  excellent  job  escaping 
and  turning  away  from  MTR435.  UAL53’s  immediate  turn  and  climb  did  not  escape 
AAL745."  The  controller  further  wrote,  "UAL53  was  slow  responding  to  instructions  and 
AAL745  was  all  over  him." 

Inconsistent  aircraft/pilot  response  rates  coupled  with  inadequate  FDADS  clarity 
aggravated  safe  resolution  of  these  blunders.  There  were  1169  aircraft  handled  with  87 
blunders  generated  into  with  the  flight  simulators.  Five  blunders  resulted  in  a  Test 
Criterion  Violation  (TCV).  A  TCV  occurred  when  controllers  could  not  prevent  two 
aircraft  from  coming  within  500  feet  of  each  other. 
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Thus,  the  controller,s  ability  to  detect  and  provide  adequate  and  safe  resolution 
instructions  was  inadequate  in  this  simulation.  This  do»  not  reflect  as  a  negative  a^>ect 
of  the  controllers’  abilities.  However,  it  indicates  the  influence  of  the  equipment  and 
simulation  guidelines  on  blunder  resolution  performance. 

In  this  simulation  configuration  air  traffic  controllers  could  not  safely  monitor  triple 
simultaneous  instrument  landing  system  approaches  using  simulated  4.8  second  data  and 
FDADS  equipment  at  a  simulated  high  density  altitude  report. 


/ 


/ 


arold  R:  Anderson 


National  Air  Traffic  Controllers  Association 
Multiple  Parallel  Approach  Program  Representative 
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APPENDIX  C 

SIMULATION  USING  THE  FDADS 
TECHNICAL  OBSERVERS  REPORT 


TECHNICAL  OBSERVERS  REPORT 
DIA  SIMULATION,  FDADS 


The  following  is  a  consensus  opinion  of  the  technical  observers 
concerning  the  new  Denver  Airport  simulation  which  began  on 
November  16,  1992. 

The  simulation  began,  as  planned,  using  the  Full  Digital  ARTS 
III  Display  System  (FDADS) .  From  the  initial  run,  it  was  quite 
evident  that  the  controllers  were  experiencing  difficulty 
recognizing  and  subsequently  resolving  aircraft  blunders. 
Although  the  clarity  of  the  indicators  and  the  definition  of  the 
targets  were  satisfactory,  determining  movement  around  the 
localizer  course  was  extremely  difficult.  The  spacing  between 
Runways  17L  and  17R  (5280  feet)  and  the  radar  update  rate  (4.8 
seconds) ,  appeared  to  be  the  two  most  prominent  factors 
responsible  for  the  controller  difficulty.  The  slightly 
increased  speeds,  due  to  the  density  altitude,  may  also  have 
given  the  controllers  some  problems.  The  spacing  between 
runways  17R  and  16  (7600  feet) ,  did  allow  the  controllers  extra 
time  in  which  to  recognize  and  take  action  to  resolve  blunders. 

Although  the  controllers  continued  to  be  extremely  vigilant, 
many  blundering  aircraft  penetrated  the  No  Transgression  Zone 
(NTZ)  before  control  instructions  were  issued  by  the  monitor 
controllers.  The  controllers  appeared  to  be  anxious  and 
somewhat  frustrated  in  their  efforts  to  monitor  these  runs. 

We  unanimously  agreed  with  the  decision  that  terminated  this 
phase  of  the  simulation  and  restarted  using  the  Final  Monitor 
Aid  (FMA)  in  place  of  the  FDADS. 


Richard  B.  Herschmann 
SATCS,  lAH  ATCT 


U/.  ^  Chr  » 

Chester  W.  Anderson 

Special  Projects  Officer 
AGL-507 


Wallace  Watson 
ATM,  HSV  ATCT 


Wai'li 
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APPENDIX  D 

SIMULATION  USING  THE  FDADS 
RISK  ASSESSMENT 


FDADS  SIMULATION  AT  DENVER 


A  demonstration  simulation  of  the  Full  Digital  Automated  Radar 
Terminal  Display  System  (FDADS)  was  conducted  November  16  and  17, 

1992.  Thirty-six  30°  blunders  were  simulated,  of  which  12  were 
shown  by  later  analysis  to  be  at-risk.  Three  TCV's  were  found 
among  the  12  at-risk  blunders,  resulting  in  an  observed  TCV  rate 
of  1/4.  After  two  days  of  simulation,  the  decision  was  made  by 
FAA  management  to  discontinue  the  FDADS  simulation  in  favor  of 
the  FMA.  The  analysis  presented  here  will  support:  that  decision. 
The  analysis  will  show  that  the  TCV  rate  for  the  FDADS  simulation 
is  significantly  larger  than  the  TCV  rate  for  the  FMA  simulation. 
The  analysis  will  also  show  that  the  larger  TCV  rate  of  the  FDADS 
simulation  could  lead  to  an  unacceptably  large  risk. 

Although  the  sample  size  is  small,  some  conclusions  regarding  the 
data  may  be  drawn.  Since  the  number  of  at-risk,  worst-case 
blunders  was  much  larger  for  the  FMA  simulation,  the  observed 
Bernoulli  ratio  for  the  FMA  simulation  may  be  regarded  as  much 
more  accurate  than  that  for  the  FDADS  simulation.  The  observed 
ratio  for  the  FMA  simulation  was  2  TCV's  per  186  at-risk 
blunders,  or  1/93.  Since  binomial  probabilities  are  easily 
computed,  the  null  hypothesis  that  the  probability  of  a  TCV 
during  the  FDADS  simulation  is  the  same  as  that  for  FMA,  Ho:p  = 
1/93,  may  be  tested  against  the  alternate  hypothesis,  Hi:p  > 

1/93,  directly  from  the  binomial  distribution.  A  significance 
level  of  0.05  will  used  to  lessen  the  likelihood  of  a  type  II 
error . 

If  the  probability  of  a  TCV  during  the  FDADS  simulation  was  also 
p  =  1/93,  then  the  probability  P  of  observing  3  TCV's  in  a  sample 
of  12  at-risk  blunders  would  be  given  by: 


=  0.000248. 


The  probability  of  3  or  more  TCV's  in  a  sample  of  12  may  also  be 
computed  and  is  found  to  be  P  =  0.000254.  Since  the  probability 
of  3  or  more  TCV's  in  a  sample  of  12  is  smaller  than  0.05,  the 
alternate  hypothesis,  that  the  probability  of  a  TCV  using  FDADS 
is  larger  than  the  probability  of  a  TCV  using  FMA,  is  accepted  as 
being  true. 


If  the  upper  confidence  limit  for  the  FMA  simulation,  5  TCV's  per 
102  at-risk  blunders,  is  used  for  the  estimate  of  the  probability 
of  a  TCV  given  an  at-risk  blunder,  then  the  null  hypothesis  would 
be  Hq:p  5/102  and  the  alternate  hypothesis  would  be  H^xp  > 
5/102.  The  probability  of  3  TCV's  in  a  sample  of  12  at-risk 
blunders,  assuming  p  =  5/102  is  computed  as  follows: 
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The  probability  of  3  or  more  TCV's  in  a  sample  of  12  at-risk 
blunders  may  also  be  computed  and  is  found  to  be  P  =  0.0186. 

Since  this  probability  is  less  than  the  chosen  level  of 
significance,  the  alternate  hypothesis,  that  the  probability  of  a 
TCV  using  FDADS  is  larger  than  the  probability  of  a  TCV  using 
FMA,  is  accepted  as  true. 

Assuming  a  binomial  distribution  and  a  0.99  confidence  interval, 
the  actual  TCV  rate  could  be  as  high  as  0.66.  This  rate  means 
that  about  100  TCV's  per  151  worst  case  blunders  could  be 
expected  while  using  the  FDADS  for  monitoring  purposes.  The 
length  of  a  confidence  interval  is  dependent  on  sample  size  and 
will  decrease  as  sample  size  is  increased.  An  observed  rate  of 
1/4  and  a  much  larger  sample  size  would  result  in  an  estimate  of 
the  upper  confidence  interval  which  would  be  smaller  than  0.66. 
However,  a  larger  sample  could  also  result  in  a  larger  observed 
ratio  which  would  in  turn  produce  a  large  upper  confidence 
interval  limit.  Therefore,  the  decision  was  made  to  use  the  upper 
confidence  interval  limit  as  an  estimate  of  the  ratio  of  TCV's  to 
worst  case  blunders. 

The  formula  which  has  been  developed  to  predict  the  acceptable 
blunder  rate  which  would  result  in  an  operational  risk  no  larger 
than  the  target  risk  value  of  4  x  10"®,  would  become: 

1  ACC  8WCB  151  algnWCB  3  app 
25  mill  app  1  algn  WCB  100  TCV  1  triple 
1  TCV  100  30°  Bl  1  30°  Blunder 
2ACC^  IWCB  “  13,797  triple  app 

Because  of  the  current  lack  of  knowledge  of  the  blunder  rate, 
this  blunder  rate  should  be  considered  questionable  and 
therefore,  unacceptable.  If,  as  has  been  suggested,  the  rate  of 

WCB's  to  30°  blunders  could  be  as  high  as  1/10,  the  result  of  the 
formula  above  would  be: 


1  30*  Blunder 
137,970  triple  app 

The  actual  blunder  rate  is  generally  assumed  to  be  much  larger 
than  this  rate  and  therefc  *e,  this  rate  should  also  be  considered 
unacceptable.  In  either  ise,  the  simulation  using  the  FDADS 
system  should  be  considered  to  have  resulted  in  an  unacceptable 
operational  risk. 

In  contrast  to  the  FDADS  simulation  was  the  FMA  simulation.  The 
FMA  simulation  was  conducted  over  17  days  and  resulted  in  186  at- 
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risk  blunders,  but  only  2  TCV's.  The  actual  TCV  rate  was  only 
0.011.  The  0.99  confidence  interval  indicated  the  TCV  rate  could 
be  as  high  as  0.0^9  or  5  TCV's  per  102  worst  case  blunders. 

Using  the  estimate  of  1  worst  case  blunder  per  100  30°  blunders, 
the  acceptable  blunder  rate  for  the  FMA  simulation  becomes: 

1  30r  Blunder 
1,021  triple  app 

If  it  is  assumed  that  the  ratio  of  worst  case  blunders  to  30° 
blunders  is  1/10,  then  the  acceptable  blunder  rate  for  the  FMA 
becomes : 

1  30°  Blunder 
10,210  triple  app 

In  either  case,  the  acceptable  rate  for  the  FMA  is  about  13.5 
times  larger  than  the  acceptable  rate  for  the  FDADS,  thus 
indicating  that  the  use  of  the  FMA  will  result  in  an  operation 
with  less  risk  than  the  FDADS. 
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APPENDIX  E 


APPROACH  PLATES 


ILS  16L 


DENVER  TOWER 
127^ 

DENVER  APPROACH 

119.1 


LEETS 

OM 


SmULTANEOUS  APPROACHES 
AUTHORIZEO  VITH  17L  AND  17R 


MISSED  APPROACH  CUMB  TO  5900.  THEN  CUMBING  RIGHT 
TURN  TO  8000  VIA  HEADING  250«  AND  DVV  R233’  TO  SLOPE 


DENVER  INTERNATIONAL 
AIRPORT  (DVX) 


ELEV  5431 

16L 


CAT. 


S-ILS 


3631 1/E  asm 


FAF  TO  MAP  NM 


E- 


E-2 


ILS  17L 


DENVER  TOWER 
120^ 

DENVER 

APPROACH 


DENVER  MTERNATIONAL 
AIRPORT  (OVX) 


ILS  DME 


I 


SlMULTAIdlJS  APPROACHES 
AUTWRIZEO  VITH  16L  ANO  17R 


MISSED  APPROACH:  GUMS  TO  5900  THEN  CUM8INQ  LEFT  TURN  TO  lELEV  5431 
8000  VIA  HEADING  090* ANO  OVV  R 133*  TO  HOKERINT. 
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APPENDIX  F 

CONTROLLER  QUESTIONNAIRES 


POST-RUN  CONTROLLER  QUESTIONNAIRE 


DIA  SIMULATION 


PARTICIPANT  CODE 
PARTNER'S  CODE  _ 
RUN  NUMBER  _ 


DATE 

TIME 

RUNWAY 


Please  fill  out  this  brief  questionnaire  on  the  run  you  have  lust 


1.  Rate  the  level  of  activity  required  during  this  run. 


1  2 
Minimal 


3  4 

Moderate 


5 

Intense 


2.  Rate  the  level  of  stress  experienced  during  this  run. 

1  2  3  4  5 

Minimal  Moderate  Intense 


3.  Rate  the  mental  effort  required  during  this  run. 

1  2  3  4  5 

Low  Acceptable  Moderately  High  Maximum 

High 


4.  Please  describe  any  unusual  occurrences  (problems  with 

visuals,  communications,  aircraft  performance,  etc.)  from  the 
last  hour.  Please  note  any  unusually  long  delays  or  incorrect 
pilot  responses. 


NOTE: 


POST-SIMULATION  CONTROLLER  QUESTIONNAIRE 
DIA  SIMULATION 


All  responses  provided  from  the  following  questionnaire 
will  be  reported  as  an  aggregate.  Individual  responses 
will  not  be  reported.  To  ensure  complete  anonymity, 
please  do  not  write  your  name  or  controller  letter  on 
the  questionnaire.  Thank  you. 
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POST-SIMULATION  CONTROLLER  QUESTIONNAIRE 
DIA  SIMULATION 


Using  the  Final  Monitor  Aid  (FMA) ,  how  difficult  was  it  to 
perform  the  Monitor  Controller  task?  Explain  and  identify  any 
additional  information  needed,  if  any,  to  perform  the  Monitor 
Controller  task? 


1  2 
Very 

Difficult 


3  4 

Average 
Difficulty 


5 

Very 

Easy 


Triple  independent  IFR  approaches  to  runways  spaced  5280  and 
7600  ft  apart  can  be  safely  conducted  as  simulated.  Explain. 


12  3 

Disagree  Neutral 


4  5 

Agree 


Except  for  deliberately  introduced  blunders,  how  realistic  was 
this  traffic  (aircraft  types,  density)?  Explain. 


1 


2 


3  4 


5 


Not 

Realistic 


Borderline 


Very 

Realistic 


4.  What  specific  strategy  or  agreement  did  you  develop  regarding 
inter-controller  coordination?  Please  describe  briefly  below 
even  if  the  arrangement  was  unspoken.  Be  specific  and  include 
controller  letter  codes. 


5.  What  specific  central  strategies,  if  any,  did  you  develop  due  to 
the  high  density  altitude  environment? 


6.  Based  on  your  experiences  during  this  simulation,  please  comment 
on  any  potential  effects  high  density  altitude  may  have  on  the 
runway  configuration  simulated? 


8.  Please  describe  any  items  in  the  simulation  which  you  believe 
were  not  realistic  or  whose  realism  could  have  been  improved 
upon  (include  any  comments  i.e  equipment,  displays, 
communication,  etc.): 
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CONTROLLER  TEST  CRITERION  VIOLATION  STATEMENT 

DIA  Simulation 

Controller  _  Runway  _  Run  #  _ 

Blunder  ID/Rwy  _  Evader  ID/Rv^  _ 


1.  Were  you  aware  a  blunder  situation  was  developing? 
(  )  Yes  (  )  No 

Explain  _ _ _ 


2.  Did  you  contemplate  taking  corrective  action? 
(  )  Yes  (  )  No 

Explain  _ 


3.  Did  you  attempt  to  take  corrective  action? 
(  )  Yes  (  )  No 

Explain  _ 
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4.  Identify  which  of  the  following  alerted  you  to  the  occurrence: 


(  )  Self-Identified  (  )  Pilot 

(  )  Other  Controller  (  )  Other 


5.  Were  you  distracted  by  anything  which  influenced  the  occurrence 
(presence  of  visitors,  speaker  volume,  loud  talking  from  others, 
etc . ) ? 

(  )  Yes  (  )  No 

Explain  _ 


6,  Brief  explanation  of  traffic  complexity; 


II.  CAUSAL  FACTORS/RECOMMENDATIONS;  Identify  any  of  the  following 
which  you  believe  contributed  to  the  incident: 

1.  Operational  Factors; 

(  )  Equipment  (  )  Pilot  Action  (  )  Oversight 

(  )  Traffic  Volume  (  )  Lack  of  Cooperation  (  )  Other 

Explain  all  the  items  checked: _ 
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III.  STATEMENT; 


1.  From  your  knowledge  of  the  incident,  provide  a  narrative 
summary: 


2.  To  prevent  recurrence  of  a  similar  operational  error/deviation, 
I  recommend  the  following: 
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PILOT  QUESTIONNAIRES 


PILOT  BREAK-OUT  QUESTIONNAIRE 


DIA  Simulation 


1.  WAS  THE  BREAK-OUT  INSTRUCTION  COMMUNICATED  CLEARLY  AND 
CONCISELY?  (e.g.,  rate  of  speech,  clarity,  volume,  etc...) 

Yes  _  No  _ 

If  no,  state  reason:  _ 


2.  HOW  WAS  THE  BREAK-OUT  INSTRUCTION  GIVEN? 

1  -  Heading,  Altitude  (in  one  transmission) 

2  -  Altitude,  Heading  (in  one  transmission) 

3  -  Heading,  Altitude  (in  two  separate  transmissions) 

4  -  Altitude,  Heading  (in  two  separate  transmissions) 

5  -  Other 

If  5  -  Other,  please  describe:  _ 


3.  WAS  A  SECOND  TRANSMISSION  REQUIRED  IN  ORDER  TO  RECEIVE  A 
COMPLETE  BREAK-OUT  INSTRUCTION? 

Yes  _  No  _ 

If  yes,  state  reason:  _ 


4.  GIVEN  THE  CONTROLLER  INSTRUCTION,  AIRCRAFT  CONFIGURATION,  AND 
FLIGHT  REGIME,  RATE  THE  DIFFICULTY  OF  THE  BREAK-OUT  MANEUVER. 


1  2  3 

Not  Difficult  Average 

Please  explain:  _ 


4  5 

Very  Difficult 


5.  WHAT,  IF  ANY,  ADDITIONAL  COMMENTS  DO  YOU  HAVE? 


FLIGHTCREW  OPINION  SURVEY 
DIA  Simulation 


DATE: 


SITE:  AVIA  BOEING  DELTA/B737  DELTA/MD88  FAATC  NASA  TWA 
PILOT  LETTER:  _ 


TOTAL  NUMBER  YEARS  FLIGHT  EXPERIENCE: 


Answer  each  question  to  the  best  of  your  ability  using  the 
scoring  scheme  shown.  You  are  invited  to  provide  additional 
comments  on  any  item  in  the  space  provided  at  the  end  of  the 
survey  form.  Please  reference  the  item  number. 

1  -  strongly  agree 

2  -  agree 

3  -*  neutral 

4  -  disagree 

5  -  strongly  disagree 

1.0  SURVEY  ITEMS 

1.1  Current  parallel  runway  procedures  require  1000  ft  of 
vertical  separation  at  the  localizer  turn-on  for 
separation. 

In  the  event  that  one  (or  both)  aircraft 
overshoot  the  localizer,  1000  ft  of 
vertical  separation  would  provide 
an  acceptable  safety  margin  provided 
aircraft  maintain  their  assigned 
altitude  until  established  on  the 
localizer  course. 

1.2  To  emphasize  the  importance  of  a  quick 
response  from  the  threatened  aircraft, 
special  phraseology  should  be  used  for 
the  break-out  maneuver. 

1.3  Given  the  premise  that  the  resolution  of  a  conflict 
situation  is  primarily  dependent  on  monitor  controller  to 
pilot  communications,  please  respond  to  the  following 
statements . 

An  alternate  communication  frequency  12345 

would  be  useful. 

Breakout  instructions  should  be  broadcast 
simultaneously  on  the  ILS  frequency.  12345 

(voice  on  the  localizer) 


1  2  3  4  5 


1  2  3  4  5 


G-2 


2 . 0  GENERAL  COMMENTS 


2.1  Please  provide  any  suggestions  on  types  of  training  to  be 
required,  types  of  certification  (etc.)  that  you  think 
would  enable  the  operation  of  multiple  simultaneous 
parallel  ILS  approaches  to  be  a  safe  and  effective 
procedure . 


2.2  Please  provide  any  suggestions  on  types  of  special 
terminology/phraseology/ instructions  for  an  evasive 
maneuver  (e.g.,  *' Immediately, '•  "Escape,"  call  sign 

emphasis  by  repeating  callsign  2  times,  altitude 
instruction  before  heading  instruction,  published  escape 
maneuver) . 


THANK  YOU  FOR  YOUR  PARTICIPATION  AND  ALL  COMMENTS  GIVEN. 
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IF  YOU  WOULD  LIKE  A  COPY  OF  THE 
DIA  SIHULATION  FINAL  TEST  REPORT, 
PLEASE  COMPLETE  THE  FOLLOWING: 


NAME:  _ 

ADDRESS:  _ 

PHONE :  _ 

NOTE:  IT  TAKES  APPROXIMATELY  1  YEAR  FOR  A  TEST 

REPORT  TO  BE  PUBLISHED  AFTER  A  SIMULATION  HAS 
BEEN  COMPLETED. 
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APPENDIX  H 


CONTROLLER  REPORT 


SIMULATION  REPORT  FOR  TRIPLE  SIMULTANEOUS  PARALLEL  ILS 
APPROACHES  FOR  THE  NEW  DENVER  INTERNATIONAL  AIRPORT  USING 
THE  FINAL  MONITOR  AID. 


This  simulation  involved  air  traffic  controllers  from  the  Denver  TRACON  as  well  as 
controllers  from  other  airports  that  conduct  Simultaneous  Instrument  Landing  System 
(ILS)  approaches.  Conditions  of  the  new  Denver  International  Airport  were  simulated. 
The  evaluation  period  was  from  November  18, 1992  to  December  15, 1992  at  the 
Federal  Aviation  Administration  Technical  Center  (FAATC).  This  simulation  evaluated 
the  effects  of  high  density  altitude  on  an  air  traffic  controller’s  ability  to  detect  and 
resolve  blunders  for  the  final  monitor  position.  A  blunder  is  defined  from  the  simulation 
documents  as  an  unexpected  turn  by  an  aircraft  already  established  on  the  localizer 
toward  another  aircraft.  The  monitor  equipment  was  the  Final  Monitor  Aid  (FMA). 

The  FMA  description  from  the  same  documents  is  a  20x20  inch,  high  resolution  color 
display  specifically  designed  for  the  monitor  controller  position.  These  displays  utilize 
2048-line  by  2048-pixel  resolution  television  raster  scan  technology.  They  incorporate  the 
controller  alert  system  hardware/software  which  is  used  in  the  precision  runway  monitor 
(PRM)  system.  FMA  alert  features  are  a  voice  activated  warning  and  data  block  color 
change  on  the  display.  This  feature  is  activated  when  an  aircraft  approaches  and/or 
enters  the  NTZ. 

Controller  resolution  was  considered  adequate  if  the  resolution  did  not  result  in  a  Test 
Criteria  Violation  (TCV).  A  TCV  is  when  controllers  could  not  prevent  two  aircraft 
from  coming  within  500  feet  of  one  another.  This  simulation  had  two  TCVs  which  gave 
the  simulation  results  of  98.9%  air  traffic  controller  accuracy  in  resolving  blunders. 

The  FMA  greatly  reduced  controller  workload  by  enhancing  the  controllers  ability  to 
resolve  blunders  and  often  discuss  different  control  instructions  as  they  were  able  to 
coordinate  control  actions  with  each  other. 

In  this  simulation  configuration  the  effects  of  high  density  altitude  were  negligible  on  the 
controllers  ability  to  detect  and  provide  adequate  resolution  instructions.  As  a  result,  the 
air  traffic  controller  team  could  safely  monitor  triple  simultaneous  ILS  approaches  for 
the  simulated  conditions  of  the  new  Denver  International  Airport  with  the  FMA. 


Harold  R-  Anderson  ATCS 

National  Air  Traffic  Controllers  Association  Representative 
to  the  Multiple  Parallel  Approach  Program 
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APPENDIX  I 

TECHNICAL  OBSERVER  REPORT 


Technical  Observers  Report 


This  report  is  the  consensus  of  the  Technical  Observers  concerning 
the  DIA  triple  simultaneous  approach  simulation.  It  contains 
general  conclusions  concerning  the  conduct  of  the  simulation,  as 
well  as  our  recommendations  to  the  Technical  Work  Group. 


The  simulation  was  to  triple  runways  spaced  7,600  feet  and  5,280 
feet  apart  with  a  simulated  radar  system  Including  a  4.8  second 
update  rate  and  the  Final  Monitor  Aid  (FMA).  FMA  is  the  Sony  20  x 
20  high  resolution  color  display  with  controller  alerts. 


During  the  simulation,  the  controllers  had  little  difficulty 
detecting  and  resolving  blunders.  The  relative  ease  in  which  the 
controllers  were  able  to  maintain  required  separation  became  more 
evident  as  they  grew  accustomed  to  the  equipment  and  procedures. 
In  fact,  as  controllers  gained  experienced  with  the  FMA,  they  were 
able  to  wait  longer  to  evaluate  the  situation  and  then  initiate 
action  to  resolve  the  blunder.  Alert  algorithms  and  runway  spacings 
appeared  to  be  the  factors  which  allowed  the  controllers  to  take 
extra  time  when  evaluating  these  situations.  The  slower  response 
time  did  not  appear  to  cause  any  “close  calls"  or  test  criterion 
violations  (TCV).  As  the  simulation  progressed,  it  appeared  that 
some  controllers  grew  bored  as  they  monitored  approaches.  Although 
the  time  spent  on  position  monitoring  simultaneous  approaches 
varies  facility  to  facility,  controllers  would  never  be  required 
to  perform  this  function  for  such  long  periods  in  reality.  The  ease 
In  which  blunders  can  be  detected  with  the  equipment  tested  and  the 
monotony  of  the  task,  are  factors  which  may  possibly  Influence  a 
controllers  ability  to  maintain  separation  between  aircraft. 
Whether  a  controller  Is  bored  or  not  is  of  course  impossible  to 
observe,  however  the  Technical  Observers  agree  this  area  may  need 
some  additional  study  through  human  factors  testing. 


All  but  two  controllers  who  participated  had  prior  experience  in 
the  procedures  and  equipment  used  for  this  test.  Early  in  the 
simulation  the  experience  level  of  the  new  controllers  was  evident. 
Even  though  a  briefing  was  conducted  to  all  personnel  and  practice 
runs  were  performed  prior  to  the  actual  simulation,  the  Technical 
Observers  believe  that  more  time  should  be  devoted  to  training  of 
controllers  new  to  these  simulations.  The  practice  runs  were 
conducted  until  the  new  controllers  indicated  they  felt  comfortable 
with  equipment  and  procedures.  The  Technical  Observers  believe  the 
training  should  be  conducted  for  a  predetermined  period  decided  by 
the  TWG  and  not  left  up  to  the  discretion  of  the  new  controllers. 
This  training  could  effect  the  early  portion  of  the  simulation.  As 
an  example,  there  were  several  nuisance  breakouts  during  the  early 
portion  of  the  simulation  due  to  the  new  and  returning  controllers 
adjusting  to  the  equipment. 


The  Technical  Observers  believe  that  consideration  should  also  be 
given  to  rotating  new  controllers  into  the  simulations.  Reaction 
tine  could  oe  affected  due  to  their  familiarity  of  the  process.  We 
believe  that  half  of  the  controller  work  force  should  be  new  to 
each  simulation. 


Communications  between  the  simulator  pilots  and  the  controllers 
was  a  factor  as  the  TCV  statements  will  reflect.  The  pilots  flying 
the  simulators  may  have  flown  for  a  particular  carrier  for  a  long 
period  of  time  but  during  the  simulation  they  were  assigned  another 
carrier  call  sign.  This  resulted  in  a  missed  communication  and 
possibly  a  TCV.  Data  will  be  reviewed  to  determine  if  this  was 
actually  the  case.  There  were  other  problems  associated  with 
communications  between  the  controllers  and  the  pseudo  pilots  due 
to  inexperience  of  the  pseudo.  These  problems  were  discussed  as 
practical  on  an  individual  basis  and  should  be  considered  when 
evaluating  simulation  results. 


Aircraft  types  and  performance  characteristics  were  always  a 
concern  to  the  controller  during  a  blunder  situation.  These 
concerns  seemed  to  be  amplified  when  a  lower  performance  aircraft 
was  Involved  and  on  the  center  runway  approach.  Although  a  TCV  may 
not  have  resulted  due  to  this  situation,  the  Technical  Observers 
suggest  that  lower  performance  aircraft  be  assigned  runway  16L  at 
DVX  when  practical  because  of  the  additional  space  between  runways. 
This  would  allow  more  time  for  the  lower  performance  aircraft  to 
evade  should  it  become  necessary. 


CONCLUSION 

The  Technical  Observers  agree  that  triple  simultaneous  approaches 
at  the  DVX  airport  can  be  safely  accomplished  using  the  4.8  second 
update  radar  and  the  Final  Monitor  Aid  (FMA). 


Richard  B.  Herschmann 
SATCS,  I AH  ATCT 


Ooel  A.  Forrest 


Joel  A.  Forrest 
SATCS.  ATL  ATCT 
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Chester  W.  Anderson 
Special  Projects  Officer 
AGL-507 


ATM,  HSV  ATCT 


APPENDIX  J 

OBSERVER  LOGS 


CXSEPNBt  RUN  #:  COaROLLERS: 

(16L)  (17R)  (17L) 

DATE:  TRAFRC  SAMPLE: 

SIM  ill  Slow  Turn  Slow  Response  Wrong  Call  Instructions  Other 

•PME  TCV  BLUNDER  ID!  RWy  EVADER  ID  jPWY  EVADER  ID !  RW  NBO  P/S  Rate  (P/A)  Time  (P/C/A)  Sign  (P/C)  Not  Followed  Factor 


OVX  Simulation,  11/16  -  12/18  92 


DATE:  CONTROLLERS:  RUN  #: 

OBSERVER  _  (16L)  (17R)  (17L)  _  TRi^^FIC  SAMPLE: 

Clock  Sim  System  I  •  i  5 

Time  Time  Start/Fal|TCV  NBO  CST  Blunder  ID  ;  Rwy  Evader  ID  \  Rwy  Evader  ID  I  Rwy  DME  COM  cyA 
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APPENDIX  K 


OPERATIONAL  ASSESSMENT 


MULTIPLE  PARALLEL  APPROACH  PROGRAM  TECHNICAL  WORK  GROUP 
(MPAP  THG)  OPERATIONAL  ASSESSMENT 


The  simulation  of  simultaneous  approaches  to  the  proposed 
triple  parallel  runway  configuration  at  the  new  Denver 
International  Airport  (OIA)  was  conducted  at  the  Federal 
Aviation  Administration  (FAA)  Technical  Center,  Atlantic 
City  International  Airport,  New  Jersey,  from  November  18-20, 
1992,  and  November  30-December  17,  1992.  The  goals  were  to 
demonstrate  the  safety  and  feasibility  of  conducting  triple 
simultaneous  Instrument  landing  system  (ILS)  operations  to 
triple  parallel  runways. 

The  simulation  included  approximately  11,800  ILS  approaches 
in  which  two  conflicts  resulted  in  less  than  a  500-foot  (ft) 
slant  range  distance.  A  total  of  705  conflicts  of  30-degree 
turns  generated  into  flight  simulators  involved 
incommunicado  blundering  aircraft.  One  hundred  eighty-six 
of  these  705  conflicts  were  "at  risk"  if  the  controller  did 
not  intervene.  Detailed  evaluation  was  conducted  on  the  two 
situations  which  resulted  in  500  ft  or  less  slant  range 
distance.  The  closest  point  of  approach  was  computed  to 
have  a  187-ft  slant  range  distance. 

Based  on  the  established  test  criteria,  the  controllers  met 
the  simulation  objective.  The  arrival  monitor  positions  in 
the  simulation  proved  to  be  operationally  effective  and 
feasible. 

The  test  controllers  participated  in  the  simulation  as 
though  they  were  controlling  live  traffic.  Their  attention 
and  dedication  was  critical  to  the  success  of  the 
simulation. 

Based  upon  the  results  of  the  simulation,  the  TWG  believes 
that  the  proposed  triple  simultaneous  ILS  approaches  at  DIA 
are  acceptable,  achievable,  and  safe  with  the  final  monitor 
aid  (FMA)  system  and  an  appropriate  radar  system,  such  as  a 
Mode  S  monopulse  system  or  an  ASR-9  radar  system  enhanced  to 
provide  improved  target  resolution  capabilities. 
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RECOMMENDATIONS 


The  Multiple  Parallel  Approach  Program  Technical  Work  Group 
(MPAP  TWG)  recommends: 

1.  There  shall  be  one  monitor  controller  for  each  runway. 
Personnel  and  equipment  shall  be  provided  to  support  the 
procedure . 

2.  All  monitor  positions  should  be  located  together  and 
near  their  respective  arrival  and  departure  positions. 

3.  A  radar  system  with  2.7  milliradians  azimuth  accuracy 
or  better,  and  the  capability  to  resolve  two  aircraft  at  20 
nmi  separated  by  .9  degrees  or  more. 
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APPENDIX  L 

RISK  ANALYSIS  OF  BLUNDERS 
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RISK  ANALYSIS  OF  BLUNDERS 


1.0  INTRODUCTION 

The  implementation  of  multiple  parallel  approaches  will  introduce 
the  risk  of  collision  of  aircraft  on  adjacent  approaches.  Other 
possible  risks  are  conflicts  during  the  turn  on  procedure  and 
possible  runway  deviations  after  landing.  Historically  the  risk 
of  collision  of  aircraft  on  adjacent  approaches  has  been 
considered  the  most  important.  The  risk  of  turn  on  conflicts  is 
deemed  acceptably  small  due  to  large  altitude  and  lateral 
separations  as  well  as  positive  radar  guidance.  The  risk  of 
runway  deviations  is  also  deemed  acceptably  small  from  the 
absence  of  large  runway  deviations  from  historic  accident  data. 

The  risk  of  collision  of  aircraft  on  adjacent  approaches  is  due 
to  a  phenomenon  called  a  blunder.  Although  it  is  well  known  that 
aircraft,  during  normal  operations,  display  a  distribution  of 
lateral  movement  left  and  right  of  the  course  centerline,  this 
distribution  is  understood  well  enough  that  the  probability  of 
the  aircraft  being  so  far  off  course  during  normal  operations  to 
cause  a  collision  is  considered  negligible.  This  already  small 
probability  is  made  even  smaller  by  the  presence  of  air  traffic 
controllers,  equipped  with  state  of  the  art  radar,  whose  task  is 
to  monitor  only  the  final  approach  segment.  Thus  the  primary 
risk  of  collision  is  due  to  a  previously  unimportant  rare  event 
called  a  blunder. 

A  blunder  can  be  defined  as  any  significant  deviation  from  the 
course  centerline  resulting  from  pilot  error  and/or  equipment 

failure.  Blunders  are  usually  thought  of  as  deviations  of  10“  or 
more  from  centerline  and  can  be  caused  by  a  variety  of  reasons 
ranging  from  pilot  error  in  the  selection  of  the  proper  ILS 
frequency  to  failure  of  an  outboard  engine  on  an  aircraft  with 
wing  mounted  engines.  Although  no  historical  documentation  of 
blunders  exists,  conversations  with  controllers  affirm  that 
blunders  do  occur  as  rare  events.  One  known  incident,  which  was 
caused  by  the  simultaneous  failure  of  two  engines  on  the  same 
side  of  a  four  engine  aircraft,  resulted  in  an  estimated  30“ 
blunder  from  the  course  centerline.  This  knowledge  resulted  in 
an  agreement  between  the  Multiple  Parallel  Approach  Procedures 
Technical  Work  Group  and  industry  representatives  to  test  blunder 

angles  up  to  and  including  30“.  However,  simulations  have 

indicated  that  blunders  through  20“  are  easily  resolvable  and 
therefore  pose  minimal  risk  to  the  operation.  Therefore  this 

study  will  concentrate  on  the  risk  posed  by  a  30“  blunder. 
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2.0  BASIC  RISK  COMPUTATION 


2.1  DEFINITION  OF  A  TCV 

Although  the  definition  of  a  collision  of  two  aircraft  is 
obvious,  a  definition  which  can  be  used  in  mathematical  analysis 
is  not  as  obvious  and  could  be  considered  by  some  to  be 
subjective.  Since  aircraft  are  basically  long  tubular  structures 
with  various  protuberances,  it  is  possible  that  two  aircraft 
could  pass  very  close  to  one  another  without  touching.  On  the 
other  hand,  the  aircraft  could  simply  touch  wing  tips  with  the 
centers  of  gravity  far  removed.  For  this  reason  it  was  decided 
to  simply  place  the  evading  aircraft  in  the  center  of  a 
hypothetical  sphere  and  determine  whether  or  not  the  center  of 
gravity  of  the  blundering  aircraft  penetrates  that  sphere.  It 
will  be  assumed  that  such  a  penetration  would  result  in  a 
collision.  Obviously  not  all  such  penetrations  would  result  in 
collisions,  but  in  mathematical  analysis  some  simplifying 
assumptions  must  be  made.  The  choice  of  the  radius  of  this 
sphere  is  somewhat  subjective.  It  must  be  at  least  as  large  as 
the  wingspan  of  the  largest  aircraft  which  will  be  involved  in 
parallel  approach  operations,  and  it  must  be  at  least  as  large  as 
the  wingspan  of  aircraft  in  the  foreseeable  future.  For  these 
reasons  the  radius  of  the  sphere  was  chosen  to  be  500  feet. 

Since  an  incursion  of  the  blundering  aircraft  into  the  500  foot 
sphere  of  the  evader  aircraft  does  not  guarantee  a  collision,  it 
will  be  called  a  Test  Criterion  Violation  or  TCV. 

2.2  DEFINITION  OF  WORST  CASE  BLUNDER 

Previous  studies  as  well  as  the  current  study  indicate  that 
blunders  in  the  30®  range  are  the  most  likely  to  result  in  a  TCV. 
The  probability  of  a  TCV  during  a  20®  or  less  blunder  is 
considered  to  be  remote.  Not  all  30®  blunders  will  result  in  a 
flight  by  the  blundering  aircraft  through  the  NTZ  into  the  path 
of  the  evading  aircraft  just  as  not  all  swerves  by  automobiles 
toward  the  center  median  result  in  a  crossing  of  the  median. 
Simulations  have  shown  that  if  the  pilot  of  the  blundering 
aircraft  is  able  to  return  the  aircraft  to  the  course  centerline 
because  of  controller  intervention  or  personal  initiative,  the 
risk  of  collision  is  negligible.  Therefore,  a  worst  case  blunder 

(WCB)  is  defined  to  be  a  30®  blunder  in  which  the  pilot  of  the 
blundering  aircraft  is  unable  to  respond  to  a  controller 
direction  to  return  to  course.  The  reason  why  the  pilot  may  not 
respond  could  be  a  communications  failure,  a  mechanical  failure, 
a  severe  weather  problem  such  as  a  thunderstorm,  or  a 
physiological  problem  of  some  member  of  the  crew.  For  the 
purposes  of  this  study,  the  reason  will  be  simply  referred  to  as 
no  response. 


2.3  BASIC  RISK  EQUATION 


A  TCV  will  occur  when  two  aircraft  are  aligned  in  such  a  way  that 
a  TCV  is  possible  and  simultaneously  a  WCB  occurs.  Intuition 
suffices  to  prove  that  an  alignment  window  exists  during  which  a 
TCV  is  possible  if  a  worst  case  blunder  occurs.  If  the  aircraft 
are  not  within  this  alignment  window  then  the  blundering  aircraft 
would  pass  harmlessly  ahead  or  behind  of  the  evading  aircraft 
without  any  evasive  movement  of  the  evading  aircraft.  Likewise, 
it  is  obvious  that  a  TCV  can  only  occur  when  a  blunder  turns  into 
a  worst  case  blunder.  Hence  a  TCV  can  only  occur  when  the 
aircraft  are  properly  aligned  during  a  blunder  which  results  in  a 
worst  case  blunder.  In  mathematical  set  theory,  this  means  that 
the  set  of  TCV's  is  a  subset  of  the  intersection  of  the  set  of 

aligned  approaches  with  the  set  of  3 O'*  blunders  and  the  set  of  no 
response  blunders. 

Although  a  TCV  does  not  necessarily  result  in  a  collision,  for 
simplicity  and  in  order  to  equate  the  probability  of  a  TCV  to  the 
existing  accident  rate,  it  will  be  assumed  that  a  TCV  will  result 
in  a  collision  and  that  a  collision  will  result  in  the  loss  of 
both  aircraft.  Therefore,  in  order  to  simplify  the  analysis,  a 
TCV  will  be  assumed  to  result  in  two  fatal  accidents. 

Using  the  notation  P (event)  to  indicate  the  probability  that  an 
event  will  occur  and  P( event  1  |  event  2)  to  indicate  the 
probability  that  event  1  will  occur  given  that  event  2  has 
already  occurred,  the  discussion  above  indicates  that  the 
probability  of  a  collision  may  be  written  as: 

P (collision)  =  P(TCV) 

=  P(TCV  and  aligned  and  WCB  and  blunder) 

=  P(TCV  I  aligned  and  WCB  and  blunder)  x 
P (aligned  |  WCB  and  blunder)  x 
P(WCB  I  blunder)  x  P (blunder) . 

In  order  to  compute  the  probability  of  a  collision  or  TCV  it  is 
necessary  to  compute  or  estimate  four  factors.  The  first  factor, 
P(TCV  I  aligned  and  WCB  and  blunder)  may  be  estimated  from  data 
collected  during  the  simulation  of  this  study.  The  simulation 
is  designed  to  determine  the  probability  that  a  TCV  will  occur 
when  an  aligned  WCB  occurs.  The  second  factor,  P (aligned  I  WCB 
and  blunder)  may  be  estimated  by  analytical  means.  The  third 
factor,  P(WCB  I  blunder)  is  not  easily  estimated,  but  bounds  may 
be  placed  on  its  possible  variation.  The  fourth  factor, 

P (blunder)  is  even  more  difficult  to  estimate  since  it  is 
extremely  small.  Since  P(TCV)  depends  on  two  factors  whose 
estimation  is  in  doubt,  it  is  desirable  to  eliminate  at  least  one 
of  the  doubtful  factors.  From  historical  data  the  probability  of 


L-3 


a  fatal  accident  during  an  approach  under  instrument 
meteorological  conditions  may  be  determined  and  used  to  find  an 
acceptable  risk  of  a  TCV.  In  this  way,  the  only  missing  variable 
in  the  equation  would  be  P (blunder).  The  formula  could  then  be 
used  to  solve  for  P (blunder) .  Thus  P(blunder)  is  given  by: 

P(blunder)  =  P (TCV) / (P (TCV | aligned  and  WCB  and  blunder) 

X  P(aligned| WCB  and  blunder)  x  P(WCB j blunder) ) . 

The  value,  P (blunder)  would  not  represent  the  actual  value  of  a 
blunder  since  that  value  is  unknown,  instead,  it  would  represent 
a  blunder  probability  which  the  system  could  tolerate  and  which 
would  provide  an  acceptable  level  of  risk  represented  by  P(TCV) . 

A  large  value  of  P (blunder)  would  be  desirable  since  it  would 
indicate  the  system  could  tolerate  a  large  blunder  probability 
and  still  meet  the  acceptable  risk  level,  P(TCV) .  A  very  small 
value  of  P (blunder)  would  be  undesirable  since  it  is  known  that 
the  actual  blunder  rate  is  small,  but  the  order  of  magnitude  is 
in  question. 

3.0  DETERMINATION  OF  ACCEPTABLE  RISK 

3.1  PHASES  OF  FLIGHT 

In  order  to  find  an  acceptable  probability  of  an  accident  due  to 
a  collision  of  aircraft  on  adjacent  approaches,  a  general 
systems  approach  to  the  overall  flight  operation  is  discussed 
first.  The  flight  operation  is  defined  to  be  the  entire  sequence 
of  events  in  a  flight  from  starting  the  engine (s)  in  preparation 
for  departure  to  shutting  down  the  engine (s)  at  the  destination. 
The  sequence  of  events  can  be  defined  with  varying  degrees  of 
detail;  however,  for  the  purposes  of  this  discussion,  the 
following  sequence  of  events  seems  appropriate: 

1.  Start  and  taxi 

2.  Take-off 

3.  Climb  to  cruise  altitude 

4.  Cruise  en  route 

5.  Descent  and  initial  approach 

6.  Final  approach 

7.  Landing,  roll-out,  taxi,  shutdown. 

This  sequence  was  chosen  because  historical  accident  data  is 
reported  using  this  sequence. 

3.2  ESTIMATING  PHASE  RISKS 

Using  data  made  available  by  the  National  Transportation  Safety 
Board  (NTSB)  and  the  FAA,  the  fatal  accident  rates  by  departure 
for  air-carrier  operations  for  the  years  1983  -  1988  have  been 
made  and  are  shown  in  Table  1.  The  accident  count  from  which 
these  rates  were  determined  includes  all  reported  air-carrier 
accidents.  The  accidents  reported  by  NTSB  may  or  may  not  be  due 
to  system  failures  or  pilot  errors;  for  example,  a  fatal  accident 
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involving  a  ground  crew  member  during  the  push  back  from  the 
jetway  is  reported.  For  this  reason,  only  accidents  caused  by 
system  failures  or  pilot  errors  are  used  in  the  determination  of 
the  phase  rates.  From  data  supplied  by  the  FAA,  the  number  of 
air  carrier  operations  or  departures  is  estimated  to  be  about 

33.3  million.  The  phase  rate  is  determined  by  dividing  the  number 
of  fatal  accidents  by  the  number  of  departures. 


Phase  of 

Flight 

Reported  Fatal 
Accidents 

Fatal  Accident  Rate 
(per  Approach) 

Start  and 

Taxi 

1 

2.9998  X 

10-8 

Take-off 

6 

1.7999  X 

lo-"^ 

Climb 

0 

- 

Cruise 

3 

8.9995  X 

10-8 

Descent 

1 

2.9998  X 

10-8 

Approach 

2 

5.9997  X 

10-8 

Landing 

1 

2.9998  X 

10-8 

Total 

14 

4.1998  X 

lo-'^ 

Table  1. 


3.3  ESTIMATING  FINAL  APPROACH  RISK 

Since  the  NTSB  reported  two  fatal  accidents  during  the  approach 
phase,  the  estimated  rate  for  the  final  approach  segment  is  6  x 
10"®  fatal  accidents  per  departure.  Since  most  approaches  are 
flown  in  visual  flight  conditions  using  visual  flight  rules  (VFR) 
it  is  necessary  to  adjust  the  rate  to  reflect  the  ntunber  of 
approaches  under  instrument  meteorological  conditions  using 
instrument  flight  rules  (IFR) .  The  number  of  instrument 
approaches  is  no  longer  recorded  by  the  FAA;  however,  using  data 
available  in  the  FAA  Statistical  Handbook  of  Aviation.  1970.  the 
percentage  of  precision  approaches  is  estimated  to  be  about  15%, 
the  percentage  of  non-precision  approaches  is  estimated  to  be 
about  2%,  and  the  number  of  visual  approaches  is  estimated  to  be 
about  85%.  Since  average  weather  conditions  are  assumed  to  be 
constant  through  the  years,  these  percentages  are  assumed  to 
still  be  accurate. 

Since  about  33.3  million  operations  were  recorded  for  the  years 
1983  -  1988,  the  number  of  precision  approaches  is  about  15%  of 

33.3  million  or  5  million  precision  approaches.  The  two  fatal 
accidents  reported  in  the  time  period  both  occurred  during 
precision  approaches.  This  leads  to  an  estimated  fatal  accident 
rate  for  precision  approaches  during  the  same  period  of  time  of 
4  X  lO'*^  or  about  1  fatal  accident  per  2.5  million  approaches. 
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3.4  ESTIMATING  COLLISION  RISK 


The  final  approach,  whether  precision,  non-precision,  or  VFR  is 
not  just  a  single  operation,  but  is  composed  of  several 
operations  requiring  certain  systems  for  their  successful 
completion.  In  order  to  determine  the  risk  of  collision  with  an 
aircraft  on  the  adjacent  glidepath,  it  is  necessary  to  determine 
each  operation  which  must  be  performed  and  each  system  which  must 
function  to  successfully  complete  the  approach  without  an 
accident.  The  following  is  a  list  of  events  which  could  produce 
an  accident  during  an  instrument  approach: 

1.  Collision  with  an  obstacle  during 

the  instrument  portion  of  the  approach. 

2.  Collision  with  an  obstacle  during  the 
visual  portion  of  the  approach. 

3.  Pilot  failure  due  to  mental  or 
physiological  malfunction. 

4.  Failure  of  aircraft  systems  except  engine, 
structures,  electronic. 

5.  Aircraft  structural  failure. 

6.  Engine  failure. 

7.  Failure  of  approach  guidance  electronics, 
ground  and  air. 

8.  Natural  environmental  phenomenon. 

9.  Midair  collision. 

10.  Midair  collision  with  an  aircraft  on  an 
adjacent  approach. 

Event  9  represents  a  collision  with  another  aircraft  which  is  not 
established  on  an  adjacent,  parallel  approach,  while  event  10 
represents  a  collision  with  an  aircraft  which  is,  or  has  been  (in 
case  of  a  blunder) , established  on  a  parallel  approach. 

Using  these  events  as  the  ones  which  could  produce  an  accident  if 
they  occur,  then  the  probability  of  an  accident  on  the  ILS 
approach,  P(A),  would  be: 

P(A)  =  1  -  P(A') . 

The  probability  that  an  accident  will  not  occur,  P(A')  is  the 
probability  that  event  1  does  not  occur,  P(l'),  and  event  2  does 
not  occur,  P(2'),  ...  ,  and  event  10  does  not  occur,  P(IO'). 
Assuming  that  the  events  are  independent,  the  probability  that  an 
accident  does  not  occur  is: 
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P(A')  =  P(1')P(2')P(3*)  ...  P(IO'). 

The  probability  of  an  accident  is  then  given  by: 

P(A)  =  1  -  P(1')P(2*)P(3 •)  ...  P(IO'). 

P(A)  =  1  -  (1  -  P(l))(l  -  P(2))  ...  (1  -  P(10)). 

=  P(l)  +  P(2)  +  ...  +  P(10)  +  Q, 

where  Q  represents  a  sum  of  terms  each  involving  products  of 
probabilities,  P(l)  through  P(10) .  Since  each  individual 
probability  is  very  small,  at  most  4  x  10~'^,  each  term  of  Q  must 
be  of  the  order  10~^^  or  less.  Neglecting  these  terms,  the 
probability  of  an  accident,  P(A),  is  given  by: 

P(A)  =  P(l)  +  P(2)  +  ...  +  P(10). 

Although  enormous  amounts  of  time  and  money  are  spent  on  accident 
investigations,  it  is  extremely  difficult  to  pinpoint  the  exact 
cause  of  an  accident.  Therefore,  it  is  extremely  difficult  in 
many  cases  to  determine  which  of  the  ten  causes  referred  to  above 
should  be  assigned  to  a  particular  accident.  Furthermore, 
because  of  the  rarity  of  accidents  due  to  each  of  the  causes, 
much  more  data  than  that  currently  available  would  be  necessary 
to  estimate  the  risk  of  each  of  the  ten  causes.  Therefore,  it  is 
apparent  that  estimates  of  the  ten  risks  are,  for  practical 
purposes,  impossible  and  that  a  different  approach  to  the  problem 
is  necessary. 

This  same  problem  is  encountered  in  the  design  of  an  aircraft. 
According  to  FAR  25.1309,  it  was  assvimed  arbitrarily  that  there 
are  100  potential  failure  conditions  in  an  airplane  which  could 
contribute  significantly  to  the  cause  of  a  serious  accident. 

Since  test  data,  historical  data,  or  even  theoretical  estimates 
are  unavailable  for  most  of  the  causes,  the  allowable  overall 
risk  of  a  serious  accident  was  apportioned  equally  among  these 
conditions,  resulting  in  an  allowable  risk  for  each  of  the 
failure  conditions  equal  to  1/100  of  the  total  risk. 

Since  ten  causes  of  a  fatal  accident  during  a  parallel  approach 
can  be  defined,  it  is  reasonable  to  assign  an  equal  probability 
to  each  of  the  causes.  Since  the  overall  probability  of  a  fatal 
accident  during  a  parallel  approach  is  the  sum  of  the 
probabilities  of  the  component  causes,  each  cause  should  be 
allocated  1/10  of  the  total  probability.  This  leads  to  an 
allowable  probability  of  4  x  10~^  or  1  fatal  accident  per  25 
million  approaches.  This  approach  should  lead  to  a  conservative 
risk  allowance  for  collision  since  the  risk  of  some  of  the  ten 
causes  may  be  so  small  as  to  be  insignificant.  This  means  that 
there  are  possibly  fewer  than  ten  significant  causes  so  that  the 
total  risk  could  have  been  divided  by  a  smaller  number  resulting 
in  a  larger  allowable  risk  for  collision. 
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4.0  DETESMINATION  OF  ALIGNMENT  WINDOW 

Since  a  TCV  occurs  when  the  two  aircraft  approach  within  500  feet 
of  each  other  it  seems  obvious  that  the  evading  aircraft  need  not 
be  in  an  exact  position  relative  to  the  blundering  aircraft,  but 
it  could  be  in  a  number  of  places  which  could  all  result  in  a 
TCV.  If  the  aircraft  are  traveling  at  the  same  speed,  then  it 
also  seems  obvious  that  the  evading  aircraft  must  be  several  feet 
behind  the  blundering  aircraft.  Therefore,  there  must  be  a 
minimum  distance  the  evading  aircraft  may  be  behind  the 
blundering  aircraft  and  a  maximum  distance  the  evading  aircraft 
may  be  behind  the  blundering  aircraft  which  may  result  in  a  TCV. 
If  the  evading  aircraft  is  between  these  two  limits,  it  is  said 
to  be  at-risk  and  an  at-risk  blunder  is  said  to  have  occurred. 

In  order  to  obtain  a  solution,  some  assumptions  are  necessary. 

The  aircraft  will  be  assumed  to  only  travel  in  the  plane  defined 
by  the  two  glide  slopes.  In  other  words,  the  vertical  component 
of  travel  will  be. neglected.  The  blundering  aircraft  will  be 
assumed  to  be  already  established  on  the  fixed  blunder  heading 
and  the  speeds  of'  the  two  aircraft  will  be  assumed  to  be 
constants.  The  evading  aircraft  will  be  assumed  to  travel 
straight  along  its  glide  path  with  no  deviations.  When 
determining  whether  two  aircraft  are  aligned  or  at  risk,  the 
assumption  of  an  immediate  turn  to  the  blunder  heading  can  be 
alleviated  by  computing  the  position  of  the  blundering  aircraft 
after  a  standard  rate  turn  to  the  blunder  heading  (see  section 
4.1) . 
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Figure  l 

To  begin  the  solution,  refer  to  figure  1.  The  figure  shows  two 
lines,  the  upper  line  is  the  glide  path  of  the  evading  aircraft 
while  the  lower  line  is  the  glide  path  of  the  blundering 
aircraft.  The  view  is  from  above  looking  down  toward  the  earth. 
The  x-axis  will  be  assvimed  to  be  the  glide  path  of  the  blundering 
aircraft  and  the  origin  will  be  located  at  the  point  where  the 
blunder  begins.  The  vertical  line  is  the  y-axis  and  passes 
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through  the  blundering  aircraft.  The  direction  of  travel  of  both 
aircraft  is  in  the  direction  of  the  positive  x-axis.  The  evading 
aircraft  will  be  assumed  to  be  located  a  distance  "a"  behind  the 
blundering  aircraft.  The  runway  separation  will  be  denoted  by 
"b",  the  speed  of  the  blundering  aircraft  will  be  denoted  by 
”Vb",  the  speed  of  the  evading  aircraft  will  be  denoted  by  "Vq” , 
the  blunder  angle  by  "6",  and  the  TCV  radius  will  be  denoted  by 
"R”.  If  the  time,  in  seconds,  measured  from  the  point  of  the 
blunder  is  denoted  by  "t",  then  the  vector  equations  of  the  paths 
of  the  two  aircraft  may  be  written  as  follows: 

^e  =  (  Vet  -  a)i  +  bj 

Pj,  =  (  vy^cosQ  )tl  +  (Vbsin0)tj 

The  two  vectors  thus  defined  will  each  have  their  tails  located 
at  the  origin  while  the  heads  will  trace  out  the  path  of  the 
respective  aircraft.  The  path  of  the  blundering  aircraft  will  be 

a  straight  line  inclined  at  an  angle  6  with  the  x-axis  and  the 
path  of  the  evading  aircraft  will  be  a  straight  line  parallel  to 
the  x-axis  at  a  distance  "b”  from  the  x-axis.  A  TCV  will  occur 
if  the  heads  of  the  vectors  come  within  "R”  feet  of  each  other. 
Mathematically,  this  will  happen  if 


I  Pb  -  Pa  I  <  R. 

This  is  equivalent  to  the  following  inequality: 

((Vbcos0  -  Ve)t  +  a)2  +  (Vbtsin0  -  b)2  <  r2. 

After  expansion  and  simplification  the  inequality  becomes: 

t2(vK2  +  ^/^2  _  2VbVeCos0)  +  2t(a(v]3cos0  -  Vg)  -  2Vbbsin0) 

+  (a2  +  b2  -  r2)  <0. 

This  inequality  is  quadratic  in  t  and  the  discriminant  must  be 
non-negative  if  there  are  to  be  values  of  t  which  satisfy  the 
inequality.  The  discriminant  results  in  the  following 
inequality: 

4 (a (Vbcos0-Ve) -v^bsinO) 2-4 (Vb2+Vg2_2vbVeCos0) (a2+b2-R2)  >  o 

Expansion  and  simplification  of  this  inequality  leads  to  the 
following  inequality: 

a2vij2(cos20  -  1)  -  2a[Vbbsin0(VbCos0  -  Vg)  ] 

+  Vb^b2sin20  +  (Vb^  +  Vg2  +  2VbVgCos0) (r2  -  b2)  >  o. 

This  inequality  is  quadratic  in  a  and  since  the  coefficient  of  a2 
is  clearly  negative,  its  solution  is  the  interval  between  the  two 
roots  of  the  quadratic  equation. 
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The  roots  of  the  quadratic  equation  may  be  solved  with  the 
quadratic  formula.  Substitution  into  the  quadratic  formula 
yields  the  following  equation  for  the  roots: 

^  _  b(r - cosO)  ± R>/ -  2rcos0+ 1 
sind 

where  r  =  Vg/Vb.  Note  that  by  using  the  ratio  of  the  two  speeds, 
the  units  in  which  the  speeds  are  expressed  is  unimportant  beyond 
the  fact  that  they  must  be  expressed  in  the  same  units.  Note 

also  that  the  roots  correspond  to  fixed  values  of  6  and  r.  This 
means  that  the  two  roots  which  may  be  obtained  by  using  either 
the  •'+”  or  the  in  the  place  indicated  by  the  symbol  mark 
the  ends  of  the  TCV  window  for  one  value  of  0  and  r.  For 

example,  if  0  =  30°,  r  =  1.2,  b  =  3000  ft,  and  R  =  500  ft,  the 
two  roots  will  be  a^  =1402.57  ft  and  a2  =  2605.13  ft.  This 
means  that  if  the  evading  aircraft  is  somewher'i  between  1402.57 
ft  and  2605.13  ft  behind  the  blundering  aircraf'  ,  a  TCV  will 
occur  unless  action  is  taken  by  the  controller  and  the  pilots. 

However,  this  window  only  applies  to  0  =  30°  and  r  =  1.2.  If  the 
blunder  angle  is  different  or  the  speed  ratio  is  different,  then 
different  roots  will  be  computed  giving  a  different  window. 

The  length  of  the  interval,  L,  is  given  by: 

2R>/r^  -2rco50+l 
sin0 

and  is  found  by  subtracting  the  smaller  value  a^  from  the  larger 
value  a2.  The  only  variables  present  in  the  equation  for  L  are 
R,  the  radius  of  the  TCV  sphere,  r,  the  ratio  of  evader  speed  to 

blunderer  speed,  and  0,  the  blunder  angle.  Since  the  runway 
separation  distance  b  is  not  present  in  the  equation,  the  length 
of  the  interval  is  independent  of  the  value  of  runway  separation. 
This  means  that  for  a  given  speed  ratio  and  blunder  angle,  the 
length  of  the  alignment  window  is  constant  regardless  of  the 
runway  separation.  Therefore,  widely  spaced  runways  and  closely 
spaced  runways  have  the  same  chance  for  alignment  of  the  two 
aircraft  for  a  given  blunder  angle  and  a  given  speed  ratio. 

Analysis  of  the  equation  for  L  indicates  that  for  a  fixed  blunder 
angle,  the  value  of  L  increases  as  r  is  increased.  Analysis  also 
indicates  that  for  a  fixed  value  of  r,  the  value  of  L  increases 

as  0  is  increased.  Therefore,  in  order  to  estimate  the  maximum 
value  of  L  that  would  be  encountered  during  parallel  runway 
operations,  it  is  only  necessary  to  estimate  the  largest  blunder 
angle  and  the  largest  speed  ratio  that  would  be  experienced.  If 
the  greatest  approach  speed  of  the  evading  aircraft  1'?  expected 
to  be  about  227  knots  and  the  smallest  approach  speec:  of  the 
blundering  aircraft  is  expected  to  be  about  120  knors  then  the 
maximum  value  of  r  would  be  1.89,  and  if  the  largest  blunder 
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angle  is  assumed  to  be  30°,  then  the  longest  alignment  window 
would  be  2279  feet. 

4.1  CORRECTION  FOR  THE  TURN 

The  analysis  above  assumes  that  the  blundering  aircraft  turns 
instantaneously  to  the  blunder  heading.  In  order  to  compensate 
for  this  assumption,  a  correction  is  made  to  the  blund<^  ng 
aircraft  position  at  the  point  where  the  blunder  is  ir  ted. 

The  blundering  aircraft  is  assumed  to  turn  at  3°  per  se  id  until 
the  blunder  heading  is  established.  The  radius  of  turn  is 
computed  from  the  speed  of  the  blundering  aircraft  and  the  amount 
of  movement  of  the  blundering  aircraft  is  determined  during  the 
turn.  This  movement,  together  with  the  movement  of  the  evading 
aircraft  (assumed  to  be  straight  along  its  original  course  since 
no  instruction  has  been  received  from  the  controller)  allows  the 
computation  of  the  relative  positions  of  the  two  aircraft 
following  the  turn  of  the  blunderer.  The  corrected  relative 
position  is  used  to  predict  whether  the  evader  is  aligned 
correctly  for  a  TCV. 

5.0  ESTIMATION  OF  TCV  PROBABILITY 

If  a  blunder  occurs  while  two  aircraft  are  aligned  properly  so 
that  a  TCV  could  happen,  the  avoidance  of  the  TCV  depends  upon 
the  reaction  time  of  the  pilot  and  controller  as  well  as  the 
update  rate  and  accuracy  of  the  surveillance  radar.  With 
adequate  data  obtained  from  real  time  simulations,  probability 
distributions  of  each  of  the  components  could  be  determined  with 
some  degree  of  confidence  and  combined  analytically  to  estimate 
the  probability  of  a  TCV.  However,  the  simulation  which  would 
provide  distributions  for  an  indirect  computation  of  the 
probability  of  a  TCV  can  also  be  used  for  a  direct  computation  of 
the  probability. 

Whether  the  goal  is  an  indirect  or  direct  computation  of  the 
probability  of  a  TCV,  the  number  of  simulated  blunders  which 
actually  occur  while  the  evading  aircraft  is  in  the  alignment 
window  must  be  determined.  In  other  words,  the  actual  number  of 
at-risk  blunders  must  be  counted.  This  may  be  computed  as 
indicated  above  by  knowing  the  speeds,  relative  positions,  and 
blunder  angle  at  the  time  the  blunder  is  initiated.  Since  the 
window  of  alignment  is  relatively  short  and  longitudinal 
separation  distances  are  reasonably  constant  the  probability  of  a 
TCV  diuring  an  at-risk  blunder  may  be  assumed  to  be  very  nearly 
constant.  If  the  probability  of  a  TCV  is  constant  then  the  TCV 
process  may  be  modeled  as  a  Bernoulli  process. 

In  a  Bernoulli  process,  there  are  only  two  outcomes  to  an 
experiment,  usually  called  success  and  failure.  The  probability 
of  either  success  or  failure  is  simply  the  ratio  of  the  event  to 
the  total  number  of  observations  during  the  experiment. 

Therefore,  the  probability  of  a  TCV  during  an  at-risk  blunder  may 


L-ll 


be  estimated  as  the  ratio  of  the  niimber  of  TCV's  to  the  total 
number  of  at-risk  blunders.  However,  when  an  experiment  is 
performed  from  a  known  Bernoulli  process,  such  as  flipping  a  coin 
or  rolling  a  die,  it  is  known  that  the  observed  estimate  of  the 
probability  will  differ  from  the  theoretic  or  underlying 
probability  and  that  different  experiments  will  produce  different 
estimates  of  the  same  underlying  probability.  Thus  the  ratio  of 
TCV's  to  at-risk  blunders  should  not  be  used  directly  as  the 
estimate  of  the  underlying  probability. 

From  the  theory  of  Bernoulli  processes,  confidence  intervals  of 
the  underlying  probability  may  be  determined  from  the  observed 
data.  A  confidence  interval  gives  a  measure  of  the  variation  from 
the  underlying  probability  that  may  be  observed  in  experimental 
data.  Confidence  intervals  always  have  a  probability  or 
confidence  level  associated  with  them.  A  confidence  interval 
might  be  termed  a  0.99  interval.  This  would  mean  that  if  100 
different  experiments  were  performed  to  estimate  the  underlying 
probability,  then  it  would  be  expected  that  about  99  of  the 
confidence  intervals  computed  from  the  observed  data  would 
contain  the  underlying  probability. 

Formulae  exist  for  the  computation  of  confidence  intervals  of 
Bernoulli  probabilities,  since  the  probability  of  a  TCV  is  very 
small,  the  appropriate  formulae  for  the  upper  and  lower  limits  of 
a  0.99  confidence  interval  are  as  follows: 


2c(n.y)p'(l-pr’ =0.005 

ysO 


2  c(n,  y)p’'  (1  -  p)"’’'  =  0.005 

y=k 


The  value  of  the  probability,  p,  must  be  found  using  numerical 
methods  such  as  Newton's  method  or  the  bisection  method. 

Since  in  a  computation  of  risk  conservatism  is  extremely 
Important,  the  value  associated  with  the  upper  limit  of  the 
confidence  interval  is  the  only  one  of  interest.  The  upper  limit 
of  a  0.99  confidence  interval  represents  a  value  which  is  almost 
certainly  larger  than  the  actual  underlying  probability.  Thus 
use  of  the  upper  confidence  interval  bound  will  provide  a 
conservative  estimate  of  the  actual  probability. 

6.0  DETERMINATION  OF  ACCEPTABLE  BLUNDER  RATE 

With  the  determination  of  a  target  risk,  a  method  of  determining 
the  window  of  alignment,  and  a  method  of  estimating  the 
probability  of  a  TCV  during  an  aligned  blunder,  the  acceptable 
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blunder  rate  may  be  computed.  The  speeds  used  in  the  simulation 
ranged  from  120  kts  to  227  kts.  The  120  kts  was  due  to  a 
blundering  aircraft  and  the  227  kts  was  due  to  an  evading 
aircraft.  If  is  assumed  that  either  aircraft  could  be  traveling 
at  any  speed  between  these  two  niunbers,  then  the  ratio  of  speeds 
ranges  from  120/227  =  .53  to  227/120  -  1.89.  Using  these  two 
ratios  as  the  minimum  and  maximum  speed  ratios  and  using  a 

maximum  blunder  angle  of  20°,  the  maximum  window  length  is  2279 
feet.  This  occurs  at  the  1.89  ratio.  Therefore,  the  probability 
of  correct  alignment,  assuming  3  miles  longitudinal  separation, 
is  given  by 


—  =  0.125  =  - 
3  x  6076  8 

Analysis  of  the  data  using  the  equations  derived  for  the  window 
of  risk,  indicated  that  the  number  of  at-risk  aircraft  was  186 
with  two  resultant  TCVs.  Using  a  .99  confidence  interval  to 
compute  the  upper  bound  for  the  binomial  probability,  the  upper 
bound  would  be  0.049.  This  would  lead  to  the  following  ratio  of 
TCV's  to  at-risk  aircraft: 


0.049  = 


49 

1000 


5 

102 


Another  factor  needed  is  the  ratio  of  Worst  Case  Blunders  to  30° 

blunders.  This  really  means,  the  ratio  of  30°  blunders  in  which 
the  pilot  of  the  blundering  aircraft  is  unable  to  respond  to 
instructions  by  the  controller.  In  previous  studies,  the  ratio 

of  worst  case  blunders  to  30°  blunders,  based  on  conversations 
with  controllers  and  pilots,  has  been  estimated  to  be  l/lOO. 
Recent  conversations  with  controllers  indicate  that  the  1/100 
ratio  may  be  too  large  and  that  the  actual  rate  may  be 
significantly  lower.  A  more  conservative  approach  would  be  to 
increase  the  1/100  ratio,  already  considered  conservative  by  the 
responding  controllers,  to  1/10.  Because  of  the  uncertainty  of 
the  ratio,  both  conservative  estimates  will  be  considered.  Since 
the  target  risk  is  given  in  accidents  per  approach,  factors  must 
be  introduced  to  correct  for  the  number  of  approaches  tedcing 
place  during  a  triple  approach  and  for  the  fact  that  one 
collision  is  equivalent  to  two  accidents.  The  equation  will  be 
displayed  with  appropriate  units  for  the  convenience  of  the 
reader . 

Using  1/100  as  the  estimate,  the  number  of  acceptable  blunders  to 
achieve  the  target  probability  of  4  x  10  or  1  fatal  accident  in 
25  million  approaches,  becomes 

i  ACC  8  WCB  102algnWCB  3app  ITCV  100  30*  Bl  1 30*  Bluoder 
ISmillapp  laIgnWCB  STCV  1  triple 2  ACC  1  WCB  1021  triple  approaches 
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This  is  a  very  high  rate,  which  would  not  be  approached  in  actual 
operations.  Since  the  actual  blunder  rate  seems  to  be  much 
smaller,  the  risk  of  the  operation  may  be  assumed  to  be  much 
smaller  than  the  target,  acceptable  risk. 

If  it  is  assumed  that  1/10  is  the  ratio  of  Worst  Case  Blunders  to 
30**  blunders  the  ratio  would  become 

1  30*  Blunder 
10,210  triple  approaches 

This  rate  may  be  the  same  order  of  magnitude  as  the  actual,  but 
unknown,  blunder  rate.  However,  since  this  rate  represents  an 
acceptable  rate  which  will  result  in  an  accident  rate  no  larger 
than  the  target  rate,  the  risk  of  the  operation  is  still  deemed 
acceptable. 

7.0  SUMMARY  AND  CONCLUSIONS 

Historical  accident  data  indicate  that  the  current  probability  or 
risk  of  a  fatal  accident  during  an  ILS  approach  in  instrximent 
meteorological  conditions  is  about  4  x  10~^.  Since  ten  primary 
causes  of  a  fatal  accident  during  the  approach  may  identified, 
including  a  midair  collision  with  an  aircraft  on  an  adjacent 
approach,  the  target  or  acceptable  risk  rate  may  be  determined  to 
be  4  X  10~®  or  1  fatal  accident  per  25  million  instrument 
approaches . 

The  window  of  risk  may  be  determined  analytically  for  two 
aircraft  on  adjacent  approaches  from  knowledge  of  their  relative 
positions,  their  speeds,  and  the  blunder  angle.  Using  this 
information,  the  number  of  at-risk  blunders  from  simulated  flight 
track  data  was  determined  to  be  186. 

Analysis  of  the  simulated  flight  track  data  also  indicated  that 
two  TCV's  occurred.  Confidence  intervals  for  the  Bernoulli 
probability  parameter  were  used  to  conservatively  estimate  the 
probability  of  a  TCV  given  an  at-risk  blunder  to  be  0.049. 

Conversations  with  controllers  were  used  to  estimate  the  ratio  of 

worst  case  30**  blunders  to  30**  blunders  to  be  between  1/100  and 
1/10.  Using  the  ratio,  1/100,  the  acceptable  blunder  rate  was 
determined  to  be  1  30**  blunder  per  1021  triple  approaches.  Using 
the  ratio,  1/10,  the  acceptable  blunder  rate  was  determined  to  l 
30^  blunder  per  10,210  approaches.  In  either  case,  the  operation 
would  meet  the  target  risk  rate. 
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